GenAI4ELab/papercli-papers huggingface dataset:
Revising and Falsifying Sparse Autoencoder Feature Explanations
Authors: George Ma; Samuel Pfrommer; Somayeh Sojoudi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Mechanistic interpretability research seeks to reverse-engineer large language models (LLMs) by uncovering the internal representations of concepts within their activations. Sparse Autoencoders (SAEs) have emerged as a valuable tool for disentangling polysemantic neurons into more monosemantic, interpretable features. However, recent work on automatic explanation generation for these features has faced challenges: explanations tend to be overly broad and fail to take polysemanticity into consideration. This work addresses these limitations by introducing a similarity-based strategy for sourcing close negative sentences that more effectively falsify generated explanations. Additionally, we propose a structured, component-based format for feature explanations and a tree-based, iterative explanation method that refines explanations. We demonstrate that our structured format and tree-based explainer improve explanation quality, while our similarity-based evaluation strategy exposes biases in existing interpretability methods. We also analyze the evolution of feature complexity and polysemanticity across LLM layers, offering new insights into information content within LLMs' residual streams.
A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
Authors: WEI-KAI CHANG; Rajiv Khanna
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Understanding the dynamics of optimization algorithms in deep learning has become increasingly critical, especially as models grow in scale and complexity. Despite the empirical success of stochastic gradient descent (SGD) and its variants in finding solutions that generalize well, the precise mechanisms underlying this generalization remain poorly understood. A particularly intriguing aspect of this phenomenon is the bias of optimization algorithms towards certain types of minima—often flatter or simpler—especially in overparameterized regimes. While prior works have associated flatness of the loss landscape with better generalization, tools to mechanistically connect data, optimization algorithms, and the nature of the resulting minima are still limited. For instance, methods like Sharpness-Aware Minimization (SAM) have shown practical gains by explicitly promoting flatness, but lack a unified theoretical framework explaining their influence across different data structures and model architectures. In this work, we introduce a comprehensive linear stability analysis framework to dissect the behavior of optimization algorithms—SGD, random perturbations, and SAM—in neural networks, focusing particularly on two-layer ReLU models. Our approach is built upon a novel coherence measure that captures the interaction between data geometry and gradient similarity, providing new insights into why and how certain solutions are favored.
Robust Regression of General ReLUs with Queries
Authors: Ilias Diakonikolas; Daniel Kane; Mingchen Ma
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We study the task of 
agnostically learning general 
(as opposed to homogeneous) ReLUs 
under the Gaussian distribution with respect 
to the squared loss. In the passive learning setting, 
recent work gave a computationally efficient algorithm 
that uses $poly(d,1/\epsilon)$ labeled examples 
and outputs a hypothesis with error $O(opt)+\epsilon$, 
where $opt$ is the squared loss of the best fit ReLU.
Here we focus on 
the interactive setting, where the learner 
has some form of query access to the labels of unlabeled 
examples. 
Our main result is the first computationally 
efficient learner  
that uses 
$d polylog(1/\epsilon)+\tilde{O}(\min\{1/p, 1/\epsilon\})$ 
black-box label queries, 
where $p$ is the bias of the target function, and achieves error $O(opt)+\epsilon$.
We complement our algorithmic result by showing 
that its query complexity 
bound is qualitatively near-optimal, 
even ignoring computational constraints. 
Finally, we establish that query access 
is essentially necessary 
to improve on the label complexity of passive learning. Specifically, for pool-based active learning, 
any active learner 
requires $\tilde{\Omega}(d/\epsilon)$ labels, 
unless it draws a super-polynomial
 number of unlabeled examples.
Multi-head Transformers Provably Learn Symbolic Multi-step Reasoning via Gradient Descent
Authors: Tong Yang; Yu Huang; Yingbin Liang; Yuejie Chi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Transformers have demonstrated remarkable capabilities in multi-step reasoning tasks. However, understandings of the underlying mechanisms by which they acquire these abilities through training remain limited, particularly from a theoretical standpoint. This work investigates how transformers learn to solve symbolic multi-step reasoning problems through chain-of-thought processes, focusing on path-finding in trees. We analyze two intertwined tasks: a backward reasoning task, where the model outputs a path from a goal node to the root, and a more complex forward reasoning task, where the model implements two-stage reasoning by first identifying the goal-to-root path and then reversing it to produce the root-to-goal path. Our theoretical analysis, grounded in the dynamics of gradient descent, shows that trained one-layer transformers can provably solve both tasks with generalization guarantees to unseen trees. In particular, our multi-phase training dynamics for forward reasoning elucidate how different attention heads learn to specialize and coordinate autonomously to solve the two subtasks in a single autoregressive path. These results provide a mechanistic explanation of how trained transformers can implement sequential algorithmic procedures. Moreover, they offer insights into the emergence of reasoning abilities, suggesting that when tasks are structured to take intermediate chain-of-thought steps, even shallow multi-head transformers can effectively solve problems that would otherwise require deeper architectures.
Towards Generalizable Retina Vessel Segmentation with Deformable Graph Priors
Authors: Ke Liu; Shangde Gao; Yichao Fu; Shangqi Gao
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Retinal vessel segmentation is critical for medical diagnosis, yet existing models often struggle to generalize across domains due to appearance variability, limited annotations, and complex vascular morphology. We propose GraphSeg, a variational Bayesian framework that integrates anatomical graph priors with structure-aware image decomposition to enhance cross-domain segmentation. GraphSeg factorizes retinal images into structure-preserved and structure-degraded components, enabling domain-invariant representation. A deformable graph prior, derived from a statistical retinal atlas, is incorporated via a differentiable alignment and guided by an unsupervised energy function. Experiments on three public benchmarks (CHASE, DRIVE, HRF) show that GraphSeg consistently outperforms existing methods under domain shifts. These results highlight the importance of jointly modeling anatomical topology and image structure for robust generalizable vessel segmentation.
Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text
Authors: Yize CHENG; Vinu Sankar Sadasivan; Mehrdad Saberi; Shoumik Saha; Soheil Feizi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The increasing capabilities of Large Language Models (LLMs) have raised concerns about their misuse in AI-generated plagiarism and social engineering. While various AI-generated text detectors have been proposed to mitigate these risks, many remain vulnerable to simple evasion techniques such as paraphrasing. However, recent detectors have shown greater robustness against such basic attacks. In this work, we introduce \textbf{Adversarial Paraphrasing}, a training-free attack framework that universally humanizes any AI-generated text to evade detection more effectively. Our approach leverages an off-the-shelf instruction-following LLM to paraphrase AI-generated content under the guidance of an AI text detector, producing adversarial examples that are specifically optimized to bypass detection. Extensive experiments show that our attack is both broadly effective and highly transferable across several detection systems. For instance, compared to simple paraphrasing attack—which, ironically, increases the true positive at 1\% false positive (T@1\%F) by 8.57\% on RADAR and 15.03\% on Fast-DetectGPT—adversarial paraphrasing, guided by OpenAI-RoBERTa-Large, reduces T@1\%F by 64.49\% on RADAR and a striking 98.96\% on Fast-DetectGPT. Across a diverse set of detectors—including neural network-based, watermark-based, and zero-shot approaches—our attack achieves an average T@1\%F reduction of 87.88\% under the guidance of OpenAI-RoBERTa-Large. 
We also analyze the tradeoff between text quality and our attack success to find that our method can significantly reduce detection rates, with mostly a slight degradation in text quality.
Our novel adversarial setup highlights the need for more robust and resilient detection strategies in the light of increasingly sophisticated evasion techniques.
Policy Gradient Methods Converge Globally in Imperfect-Information Extensive-Form Games
Authors: Fivos Kalogiannis; Gabriele Farina
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Multi-agent reinforcement learning (MARL) has long been seen as inseparable from Markov games (Littman 1994). Yet, the most remarkable achievements of practical MARL have arguably been in extensive-form games (EFGs)---spanning games like Poker, Stratego, and Hanabi. At the same time, little is known about provable equilibrium convergence for MARL algorithms applied to EFGs as they stumble upon the inherent nonconvexity of the optimization landscape and the failure of the value-iteration subroutine in EFGs. To this goal, we utilize contemporary advances in nonconvex optimization theory to prove that regularized alternating policy gradient with (i) *direct policy parametrization*, (ii) *softmax policy parametrization*, and (iii) *softmax policy parametrization with natural policy gradient* updates converge to an approximate Nash equilibrium (NE) in the *last-iterate* in imperfect-information perfect-recall zero-sum EFGs. Namely, we observe that since the individual utilities are concave with respect to the sequence-form strategy, they satisfy gradient dominance w.r.t. the behavioral strategy---or, \textit{policy}, in reinforcement learning terms. We exploit this structure to further prove that the regularized utility satisfies the much stronger proximal Polyak- Łojasiewicz condition. In turn, we show that the different flavors of alternating policy gradient methods converge to an $\epsilon$-approximate NE with a number of iterations and trajectory samples that are  polynomial in $1/\epsilon$ and the natural parameters of the game. Our work is a preliminary---yet principled---attempt in bridging the conceptual gap between the theory of Markov and imperfect-information EFGs while it aspires to stimulate a deeper dialogue between them.
Does Thinking More Always Help? Mirage of Test-Time Scaling in Reasoning Models
Authors: Soumya Suvra Ghosal; Souradip Chakraborty; Avinash Reddy; Yifu Lu; Mengdi Wang; Dinesh Manocha; Furong Huang; Mohammad Ghavamzadeh; Amrit Singh Bedi
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent trends in test-time scaling for reasoning models (e.g., OpenAI o1, DeepSeek R1) have led to a popular belief that extending thinking traces using prompts like “Wait” or “Let me rethink” can improve performance.  This raises a natural question: Does thinking more at test-time truly lead to better reasoning? To answer this question, we perform a detailed empirical study across models and benchmarks, which reveals a consistent pattern of initial performance improvements from additional thinking followed by a decline, due to "overthinking". To understand this non-monotonic trend, we consider a simple probabilistic model, which reveals that additional thinking increases output variance—creating an illusion of improved reasoning while ultimately undermining precision. Thus, observed gains from "more thinking" are not true indicators of improved reasoning, but artifacts stemming from the connection between model uncertainty and evaluation metric. This suggests that test-time scaling through extended thinking is not an effective way to utilize the inference thinking budget. Recognizing these limitations, we introduce an alternative test-time scaling approach, parallel thinking, inspired by Best-of-N sampling. Our method generates multiple independent reasoning paths within the same inference budget and selects the most consistent response via majority vote, achieving up to 20% higher accuracy compared to extended thinking. This provides a simple yet effective mechanism for test-time scaling of reasoning models.
Alignment of Large Language Models with Constrained Learning
Authors: Botong Zhang; Shuo Li; Ignacio Hounie; Osbert Bastani; Dongsheng Ding; Alejandro Ribeiro
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We study the problem of computing an optimal large language model (LLM) policy for the constrained alignment problem, where the goal is to maximize a primary reward objective while satisfying constraints on secondary utilities. Despite the popularity of Lagrangian-based LLM policy search in constrained alignment, iterative primal-dual methods often fail to converge, and non-iterative dual-based methods do not achieve optimality in the LLM parameter space. To address these challenges, we employ Lagrangian duality to develop an iterative dual-based alignment method that alternates between updating the LLM policy via Lagrangian maximization and updating the dual variable via dual descent. In theory, we characterize the primal-dual gap between the primal value in the distribution space and the dual value in the LLM parameter space. We further quantify the optimality gap of the learned LLM policies at near-optimal dual variables with respect to both the objective and the constraint functions. These results prove that dual-based alignment methods can find an optimal constrained LLM policy, up to an LLM parametrization gap. We demonstrate the effectiveness and merits of our approach through extensive experiments conducted on the PKU-SafeRLHF and Anthropic HH-RLHF datasets.
Aligning Text-to-Image Diffusion Models to Human Preference by Classification
Authors: Longquan Dai; Xiaolu Wei; He Wang; Shaomeng Wang; Jinhui Tang
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Text-to-image diffusion models are typically trained  on large-scale web data, often resulting in outputs that misalign with human preferences. Inspired by preference learning in large language models, we propose ABC (Alignment by Classification), a simple yet effective framework for aligning diffusion models with human preferences. In contrast to prior DPO-based methods that depend on suboptimal supervised fine-tuned (SFT) reference models, ABC assumes access to an ideal reference model perfectly aligned with human intent and reformulates alignment as a classification problem. Under this view, we recognize that preference data naturally forms a semi-supervised classification setting. To address this, we propose a data augmentation strategy that transforms preference comparisons into fully supervised training signals. We then introduce a classification-based ABC loss to guide alignment. Our alignment by classification approach could effectively steer the diffusion model toward the behavior of the ideal reference. Experiments on various diffusion models show that our ABC consistently outperforms existing baselines, offering a scalable and robust solution for preference-based text-to-image fine-tuning.
Sketched Gaussian Mechanism for Private Federated Learning
Authors: Qiaobo Li; Zhijie Chen; Arindam Banerjee
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Communication cost and privacy are two major considerations in federated learning (FL). For communication cost, gradient compression by sketching the clients’ transmitted model updates is often used for reducing per‐round communication. For privacy, the Gaussian mechanism (GM), which consists of clipping updates and adding Gaussian noise, is commonly used to guarantee client‐level differential privacy. Existing literature on private FL analyzes privacy of sketching and GM in an isolated manner, illustrating that sketching provides privacy determined by the sketching dimension and that GM has to supply any additional desired privacy. 

In this paper, we introduce the Sketched Gaussian Mechanism (SGM), which directly combines sketching and the Gaussian mechanism for privacy. Using Rényi-DP tools, we present a joint analysis of SGM's overall privacy guarantee, which is significantly more flexible and sharper compared to isolated analysis of sketching and GM privacy. In particular, we prove that the privacy level of SGM for a fixed noise magnitude is proportional to $1/\sqrt{b}$, where $b$ is the sketching dimension, indicating that (for moderate $b$) SGM can provide much stronger privacy guarantees than the original GM under the same noise budget. We demonstrate the application of SGM to FL with either gradient descent or adaptive server optimizers, and establish theoretical results on optimization convergence, which exhibits only a logarithmic dependence on the number of parameters $d$. Experimental results confirm that at the same privacy level, SGM based FL is at least competitive with non‐sketching private FL variants and outperforms them in some settings. Moreover, using adaptive optimization at the server improves empirical performance while maintaining the privacy guarantees.
Correlation Dimension of Autoregressive Large Language Models
Authors: Xin Du; Kumiko Tanaka-Ishii
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Large language models (LLMs) have achieved remarkable progress in natural
language generation, yet they continue to display puzzling behaviors—such as
repetition and incoherence—even when exhibiting low perplexity. This
highlights a key limitation of conventional evaluation metrics, which
emphasize local prediction accuracy while overlooking long-range structural
complexity.  We introduce correlation dimension, a fractal-geometric measure
of self-similarity, to quantify the epistemological complexity of text as
perceived by a language model. This measure captures the hierarchical
recurrence structure of language, bridging local and global properties in a
unified framework.  Through extensive experiments, we show that correlation
dimension (1) reveals three distinct phases during pretraining, (2) reflects
context-dependent complexity, (3) indicates a model's tendency toward
hallucination, and (4) reliably detects multiple forms of degeneration in
generated text.  The method is computationally efficient, robust to model
quantization (down to 4-bit precision), broadly applicable across
autoregressive architectures (e.g., Transformer and Mamba), and provides
fresh insight into the generative dynamics of LLMs.
Joint‑Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self‑Supervised Learning
Authors: Hugues Van Assel; Mark Ibrahim; Tommaso Biancalani; Aviv Regev; Randall Balestriero
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Reconstruction and joint-embedding have emerged as two leading paradigms in Self‑Supervised Learning (SSL). 
Reconstruction methods focus on recovering the original sample from a different view in input space. On the other hand, joint-embedding methods align the representations of different views in latent space.
Both approaches offer compelling advantages, yet practitioners lack clear guidelines for choosing between them.
In this work, we unveil the core mechanisms that distinguish each paradigm. 
By leveraging closed-form solutions for both approaches, we precisely characterize how the view generation process, e.g. data augmentation, impacts the learned representations.
We then demonstrate that, unlike supervised learning, both SSL paradigms require a minimal alignment between augmentations and irrelevant features to achieve asymptotic optimality with increasing sample size. 
Our findings indicate that in scenarios where these irrelevant features have a large magnitude, joint-embedding methods are preferable because they impose a strictly weaker alignment condition compared to reconstruction-based methods. 
These results not only clarify the trade-offs between the two paradigms but also substantiate the empirical success of joint-embedding approaches on real-world challenging datasets.
Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale
Authors: James A. Michaelov; Roger P. Levy; Ben Bergen
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: We show that across architecture (Transformer vs. Mamba vs. RWKV), training dataset (OpenWebText vs. The Pile), and scale (14 million parameters to 12 billion parameters), autoregressive language models exhibit highly consistent patterns of change in their behavior over the course of pretraining. Based on our analysis of over 1,400 language model checkpoints on over 110,000 tokens of English, we find that up to 98% of the variance in language model behavior at the word level can be explained by three simple heuristics: the unigram probability (frequency) of a given word, the $n$-gram probability of the word, and the semantic similarity between the word and its context. Furthermore, we see consistent behavioral phases in all language models, with their predicted probabilities for words overfitting to those words' $n$-gram probabilities for increasing $n$ over the course of training. Taken together, these results suggest that learning in neural language models may follow a similar trajectory irrespective of model details.
Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformers
Authors: Woomin Song; Sai Muralidhar Jayanthi; Srikanth Ronanki; Kanthashree Mysore Sathyendra; Jinwoo Shin; Aram Galstyan; Shubham Katiyar; Sravan Babu Bodapati
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: As large language models increasingly gain popularity in real-world applications, processing extremely long contexts, often exceeding the model’s pre-trained context limits, has emerged as a critical challenge. While existing approaches to efficient long-context processing show promise, recurrent compression-based methods struggle with information preservation, whereas random access approaches require substantial memory resources. We introduce REFORM, a novel inference framework that efficiently handles long contexts through a two-phase approach. First, it incrementally processes input chunks while maintaining a compressed KV cache, constructs cross-layer context embeddings, and utilizes early exit strategy for improved efficiency. Second, it identifies and gathers essential tokens via similarity matching and selectively recomputes the KV cache. Compared to baselines, REFORM achieves over 50% and 27% performance gains on RULER and BABILong respectively at 1M context length. It also outperforms baselines on ∞-Bench, RepoEval, and MM-NIAH, demonstrating flexibility across diverse tasks and domains. Additionally, REFORM reduces inference time by 30% and peak memory usage by 5%, achieving both efficiency and superior performance.