GenAI4ELab/papercli-papers huggingface dataset:
LittleBit: Ultra Low-Bit Quantization via Latent Factorization
Authors: Banseok Lee; Dongkyu Kim; Youngcheon you; Young-Min Kim
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The deployment of large language models (LLMs) is frequently hindered by prohibitive memory and computational requirements. While quantization mitigates these bottlenecks, maintaining model fidelity in the sub-1-bit regime remains a persistent challenge. In this paper, we introduce LittleBit, a novel framework for extreme LLM compression. We target quantization rates as low as $0.1$ bits per weight (BPW), achieving a memory reduction of approximately $31\times$, which effectively compresses Llama2-13B to under $0.9$ GB. We represent weights via low-rank latent matrix factorization and subsequently binarize the resulting factors. To counteract the information loss inherent to such drastic precision reduction, we integrate a multi-scale compensation mechanism that learns importance parameters across row, column, and latent dimensions. Two primary contributions enable effective training: Dual Sign-Value-Independent Decomposition (Dual-SVID) for quantization-aware training (QAT) initialization, and Residual Compensation to minimize approximation errors. Extensive experiments confirm the superiority of LittleBit in the sub-1-bit domain; for instance, our method at $0.1$ BPW surpasses the performance of leading techniques operating at $0.7$ BPW on Llama2-7B. We establish a new size-performance trade-off---unlocking a potential $11.6\times$ inference speedup relative to FP16---and render powerful LLMs practical for resource-constrained environments. Our code is available at https://github.com/SamsungLabs/LittleBit.
SymMaP: Improving Computational Efficiency in Linear Solvers through Symbolic Preconditioning
Authors: Hong Wang; Jie Wang; Minghao Ma; Haoran Shao; Haoyang Liu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Matrix preconditioning is a critical technique to accelerate the solution of linear systems, where performance heavily depends on the selection of preconditioning parameters.
Traditional parameter selection approaches often define fixed constants for specific scenarios.
However, they rely on domain expertise and fail to consider the instance-wise features for individual problems, limiting their performance. 
In contrast, machine learning (ML) approaches, though promising, are hindered by high inference costs and limited interpretability. 
To combine the strengths of both approaches, we propose a symbolic discovery framework—namely, **Sym**bolic **Ma**trix **P**reconditioning (**SymMaP**)—to learn efficient symbolic expressions for preconditioning parameters. 
Specifically, we employ a neural network to search the high-dimensional discrete space for expressions that can accurately predict the optimal parameters. 
The learned expression allows for high inference efficiency and excellent interpretability (expressed in concise symbolic formulas), making it simple and reliable for deployment. 
Experimental results show that SymMaP consistently outperforms traditional strategies across various benchmarks.
Learning 3D Anisotropic Noise Distributions Improves Molecular Force Fields
Authors: Xixian Liu; Rui Jiao; Zhiyuan Liu; Yurou Liu; Yang Liu; Ziheng Lu; Wenbing Huang; Yang Zhang; Yixin Cao
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Coordinate denoising has emerged as a promising method for 3D molecular pretraining due to its theoretical connection to learning molecular force field. However, existing denoising methods rely on oversimplied molecular dynamics that assume atomic motions to be isotropic and homoscedastic. To address these limitations, we propose a novel denoising framework AniDS: Anisotropic Variational Autoencoder for 3D Molecular Denoising. AniDS introduces a structure-aware anisotropic noise generator that can produce atom-specific, full covariance matrices for Gaussian noise distributions to better reflect directional and structural variability in molecular systems. These covariances are derived from pairwise atomic interactions as anisotropic corrections to an isotropic base. Our design ensures that the resulting covariance matrices are symmetric, positive semi-definite, and SO(3)-equivariant, while providing greater capacity to model complex molecular dynamics. Extensive experiments show that AniDS outperforms prior isotropic and homoscedastic denoising models and other leading methods on the MD17 and OC22 benchmarks, achieving average relative improvements of 8.9% and 6.2% in force prediction accuracy. Our case study on a crystal and molecule structure shows that AniDS adaptively suppresses noise along the bonding direction, consistent with physicochemical principles. Our code is available at https://github.com/ZeroKnighting/AniDS.
ADPretrain: Advancing Industrial Anomaly Detection via Anomaly Representation Pretraining
Authors: Xincheng Yao; Yan Luo; Zefeng Qian; Chongyang Zhang
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: The current mainstream and state-of-the-art anomaly detection (AD) methods are
substantially established on pretrained feature networks yielded by ImageNet pre-
training. However, regardless of supervised or self-supervised pretraining, the
pretraining process on ImageNet does not match the goal of anomaly detection
(i.e., pretraining in natural images doesn’t aim to distinguish between normal and
abnormal). Moreover, natural images and industrial image data in AD scenarios
typically have the distribution shift. The two issues can cause ImageNet-pretrained
features to be suboptimal for AD tasks. To further promote the development of
the AD field, pretrained representations specially for AD tasks are eager and very
valuable. To this end, we propose a novel AD representation learning framework
specially designed for learning robust and discriminative pretrained representa-
tions for industrial anomaly detection. Specifically, closely surrounding the goal
of anomaly detection (i.e., focus on discrepancies between normals and anoma-
lies), we propose angle- and norm-oriented contrastive losses to maximize the
angle size and norm difference between normal and abnormal features simulta-
neously. To avoid the distribution shift from natural images to AD images, our
pretraining is performed on a large-scale AD dataset, RealIAD. To further alle-
viate the potential shift between pretraining data and downstream AD datasets,
we learn the pretrained AD representations based on the class-generalizable repre-
sentation, residual features. For evaluation, based on five embedding-based AD
methods, we simply replace their original features with our pretrained represen-
tations. Extensive experiments on five AD datasets and five backbones consis-
tently show the superiority of our pretrained features. The code is available at
https://github.com/xcyao00/ADPretrain.
Global Convergence for Average Reward Constrained MDPs with Primal-Dual Actor Critic Algorithm
Authors: Yang Xu; Swetha Ganesh; Washim Uddin Mondal; Qinbo Bai; Vaneet Aggarwal
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: This paper investigates infinite-horizon average reward Constrained Markov Decision Processes (CMDPs) under general parametrized policies with smooth and bounded policy gradients. We propose a Primal-Dual Natural Actor-Critic algorithm that adeptly manages constraints while ensuring a high convergence rate. In particular, our algorithm achieves global convergence and constraint violation rates of $\tilde{\mathcal{O}}(1/\sqrt{T})$ over a horizon of length $T$ when the mixing time, $\tau_{\mathrm{mix}}$, is known to the learner. In absence of knowledge of $\tau_{\mathrm{mix}}$, the achievable rates change to $\tilde{\mathcal{O}}(1/T^{0.5-\epsilon})$ provided that $T \geq \tilde{\mathcal{O}}\left(\tau_{\mathrm{mix}}^{2/\epsilon}\right)$. Our results match the theoretical lower bound for Markov Decision Processes and establish a new benchmark in the theoretical exploration of average reward CMDPs.
TabSTAR: A Tabular Foundation Model for Tabular Data with Text Fields
Authors: Alan Arazi; Eilam Shapira; Roi Reichart
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: While deep learning has achieved remarkable success across many domains, it has historically underperformed on tabular learning tasks, which remain dominated by gradient boosting decision trees. However, recent advancements are paving the way for Tabular Foundation Models, which can leverage real-world knowledge and generalize across diverse datasets, particularly when the data contains free-text. Although incorporating language model capabilities into tabular tasks has been explored, most existing methods utilize static, target-agnostic textual representations, limiting their effectiveness. We introduce TabSTAR: a Tabular Foundation Model with Semantically Target-Aware Representations. TabSTAR is designed to enable transfer learning on tabular data with textual features, with an architecture free of dataset-specific parameters. It unfreezes a pretrained text encoder and takes as input target tokens, which provide the model with the context needed to learn task-specific embeddings. TabSTAR achieves state-of-the-art performance for both medium- and large-sized datasets across known benchmarks of classification tasks with text features, and its pretraining phase exhibits scaling laws in the number of datasets, offering a pathway for further performance improvements.
GauDP: Reinventing Multi-Agent Collaboration through Gaussian-Image Synergy in Diffusion Policies
Authors: Ziye Wang; Li Kang; Yiran Qin; Jiahua Ma; zhanglin peng; LEI BAI; Ruimao Zhang
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Despite significant advances in robotic policy generation, effective coordination in embodied multi-agent systems remains a fundamental challenge—particularly in scenarios where agents must balance individual perspectives with global environmental awareness.
Existing approaches often struggle to balance fine-grained local control with comprehensive scene understanding, resulting in limited scalability and compromised collaboration quality.
In this paper, we present GauDP, a novel Gaussian-image synergistic representation that facilitates scalable, perception-aware imitation learning in multi-agent collaborative systems.
Specifically, GauDP reconstructs a globally consistent 3D Gaussian field from local-view RGB images, allowing all agents to dynamically query task-relevant features from a shared scene representation. 
This design facilitates both fine-grained control and globally coherent behavior without requiring additional sensing modalities. 
We evaluate GauDP on the RoboFactory benchmark, which includes diverse multi-arm manipulation tasks. 
Our method achieves superior performance over existing image-based methods and approaches the effectiveness of point-cloud-driven methods, while maintaining strong scalability as the number of agents increases. 
Extensive ablations and visualizations further demonstrate the robustness and efficiency of our unified local-global perception framework for multi-agent embodied learning.
Antidistillation Sampling
Authors: Yash Savani; Asher Trockman; Zhili Feng; Yixuan Even Xu; Avi Schwarzschild; Alexander Robey; Marc Anton Finzi; J Zico Kolter
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Frontier models that generate extended reasoning traces inadvertently produce token sequences that can facilitate model distillation. Recognizing this vulnerability, model owners may seek sampling strategies that limit the effectiveness of distillation without compromising model performance. *Antidistillation sampling* provides exactly this capability. By strategically modifying a model's next-token probability distribution, antidistillation sampling poisons reasoning traces, rendering them significantly less effective for distillation while preserving the model's utility.
Discovering Opinion Intervals from Conflicts in Signed Graphs
Authors: Peter Blohm; Florian Chen; Aristides Gionis; Stefan Neumann
Venue: NeurIPS 2025 (NeurIPS 2025 oral)

Abstract: Online social media provide a platform for people to discuss current events and exchange opinions with their peers. While interactions are predominantly positive, in recent years, there has been a lot of research to understand the conflicts in social networks and how they are based on different views and opinions.  In this paper, we ask whether the conflicts in a network reveal a small and interpretable set of prevalent opinion ranges that explain the users' interactions.  More precisely, we consider signed graphs, where the edge signs indicate positive and negative interactions of node pairs, and our goal is to infer opinion intervals that are consistent with the edge signs.  We introduce an optimization problem that models this question, and we give strong hardness results and a polynomial-time approximation scheme by utilizing connections to interval graphs and the Correlation Clustering problem.  We further provide scalable heuristics and show that in experiments they yield more expressive solutions than Correlation Clustering baselines. We also present a case study on a novel real-world dataset from the German parliament, showing that our algorithms can recover the political leaning of German parties based on co-voting behavior.
Controlling Thinking Speed in Reasoning Models
Authors: Zhengkai Lin; Zhihang Fu; Ze Chen; Chao Chen; Liang Xie; Wenxiao Wang; Deng Cai; Zheng Wang; Jieping Ye
Venue: NeurIPS 2025 (NeurIPS 2025 spotlight)

Abstract: Human cognition is theorized to operate in two modes: fast, intuitive System 1 thinking and slow, deliberate System 2 thinking.
While current Large Reasoning Models (LRMs) excel at System 2 thinking, their inability to perform fast thinking leads to high computational overhead and latency.
In this work, we enable LRMs to approximate human intelligence through dynamic thinking speed adjustment, optimizing accuracy-efficiency trade-offs.
Our approach addresses two key questions: (1) how to control thinking speed in LRMs, and (2) when to adjust it for optimal performance.
For the first question, we identify the steering vector that governs slow-fast thinking transitions in LRMs' representation space.
Using this vector, we achieve the first representation editing-based test-time scaling effect, outperforming existing prompt-based scaling methods.
For the second question, we apply real-time difficulty estimation to signal reasoning segments of varying complexity.
Combining these techniques, we propose the first reasoning strategy that enables fast processing of easy steps and deeper analysis for complex reasoning. 
Without any training or additional cost, our plug-and-play method yields an average +1.3\% accuracy with -8.6\% token usage across leading LRMs and advanced reasoning benchmarks.
All of our algorithms are implemented based on vLLM and are expected to support broader applications and inspire future research.
STAR: Spatial-Temporal Tracklet Matching for Multi-Object Tracking
Authors: Xuewei Bai; Yongcai Wang; Deying Li; Haodi Ping; LI Chunxu
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Existing tracking-by-detection Multi-Object Tracking methods mainly rely on associating objects with tracklets using motion and appearance features. However, variations in viewpoint and occlusions can result in discrepancies between the features of current objects and those of historical tracklets. To tackle these challenges, this paper proposes a novel Spatial-Temporal Tracklet Graph Matching paradigm (STAR). The core idea of STAR is to achieve long-term, reliable object association through the association of ``tracklet clips (TCs)". TCs are segments of confidently associated multi-object trajectories, which are linked through graph matching. Specifically, STAR initializes TCs using a Confident Initial Tracklet Generator (CITG) and constructs a TC graph via Tracklet Clip Graph Construction (TCGC). In TCGC, each object in a TC is treated as a vertex, with the appearance and local topology features encoded on the vertex. The vertices and edges of the TC graph are then updated through message propagation to capture higher-order features. Finally, a Tracklet Clip Graph Matching (TCGM) method is proposed to efficiently and accurately associate the TCs through graph matching. STAR is model-agnostic, allowing for seamless integration with existing methods to enhance their performance. Extensive experiments on diverse datasets, including MOTChallenge, DanceTrack, and VisDrone2021-MOT, demonstrate the robustness and versatility of STAR, significantly improving tracking performance under challenging conditions.
Turning the Tables: Enabling Backward Transfer via Causal-Aware LoRA in Continual Learning
Authors: Chaoyang Li; Runze Ye; Jianyang Qin; Jinhao Cui; Lingzhi Wang; Ning Hu; Qing Liao
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Current parameter-efficient fine-tuning (PEFT) methods have shown superior performance in continual learning. However, most existing PEFT-based methods focus on mitigating catastrophic forgetting by limiting modifications to the old task model caused by new tasks. This hinders backward knowledge transfer, as when new tasks have a strong positive correlation with old tasks, appropriately training on new tasks can transfer beneficial knowledge to old tasks. Critically, achieving backward knowledge transfer faces two fundamental challenges: (1) some parameters may be ineffective on task performance, which constrains the task solution space and model capacity; (2) since old task data are inaccessible, modeling task correlation via shared data is infeasible. To address these challenges, we propose CaLoRA, a novel \textbf{c}ausal-\textbf{a}ware \textbf{lo}w-\textbf{r}ank \textbf{a}daptation framework that is the first PEFT-based continual learning work with backward knowledge transfer. Specifically, we first propose \textbf{p}ar\textbf{a}meter-level \textbf{c}ounterfactual \textbf{a}ttribution (PaCA) that estimates the causal effect of LoRA parameters via counterfactual reasoning, identifying effective parameters from a causal view. Second, we propose \textbf{c}ross-t\textbf{a}sk \textbf{g}radient \textbf{a}daptation (CaGA) to quantify task correlation by gradient projection and evaluate task affinity based on gradient similarity. By incorporating causal effect, task correlation, and affinity, CaGA adaptively adjusts task gradients, facilitating backward knowledge transfer without relying on data replay. Extensive experiments across multiple benchmarks and continual learning settings show that CaLoRA outperforms state-of-the-art methods. In particular, CaLoRA better mitigates catastrophic forgetting by enabling positive backward knowledge transfer.
STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem
Authors: Hong Wang; Jiang Yixuan; Jie Wang; Xinyi Li; Jian Luo; huanshuo dong
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Operator eigenvalue problems play a critical role in various scientific fields and engineering applications, yet numerical methods are hindered by the curse of dimensionality. Recent deep learning methods provide an efficient approach to address this challenge by iteratively updating neural networks. 
These methods' performance relies heavily on the spectral distribution of the given operator: larger gaps between the operator's eigenvalues will improve precision, thus tailored spectral transformations that leverage the spectral distribution can enhance their performance. Based on this observation, we propose the **S**pectral **T**ransformation **Net**work (**STNet**).
During each iteration, STNet uses approximate eigenvalues and eigenfunctions to perform spectral transformations on the original operator, turning it into an equivalent but easier problem.
Specifically, we employ deflation projection to exclude the subspace corresponding to already solved eigenfunctions, thereby reducing the search space and avoiding converging to existing eigenfunctions. 
Additionally, our filter transform magnifies eigenvalues in the desired region and suppresses those outside, further improving performance.
Extensive experiments demonstrate that STNet consistently outperforms existing learning-based methods, achieving state-of-the-art performance in accuracy.
Bayesian Concept Bottleneck Models with LLM Priors
Authors: Jean Feng; Avni Kothari; Lucas Zier; Chandan Singh; Yan Shuo Tan
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Concept Bottleneck Models (CBMs) have been proposed as a compromise between white-box and black-box models, aiming to achieve interpretability without sacrificing accuracy. The standard training procedure for CBMs is to predefine a candidate set of human-interpretable concepts, extract their values from the training data, and identify a sparse subset as inputs to a transparent prediction model. However, such approaches are often hampered by the tradeoff between exploring a sufficiently large set of concepts versus controlling the cost of obtaining concept extractions, resulting in a large interpretability-accuracy tradeoff. This work investigates a novel approach that sidesteps these challenges: BC-LLM iteratively searches over a potentially infinite set of concepts within a Bayesian framework, in which Large Language Models (LLMs) serve as both a concept extraction mechanism and prior. Even though LLMs can be miscalibrated and hallucinate, we prove that BC-LLM can provide rigorous statistical inference and uncertainty quantification. Across image, text, and tabular datasets, BC-LLM outperforms interpretable baselines and even black-box models in certain settings, converges more rapidly towards relevant concepts, and is more robust to out-of-distribution samples.
Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solving
Authors: Hossein Askari; Yadan Luo; Hongfu Sun; Fred Roosta
Venue: NeurIPS 2025 (NeurIPS 2025 poster)

Abstract: Recent advances in *inverse problem* solving have increasingly adopted flow *priors* over diffusion models due to their ability to construct straight probability paths from noise to data, thereby enhancing efficiency in both training and inference. However, current flow-based inverse solvers face two primary limitations: (i) they operate directly in pixel space, which demands heavy computational resources for training and restricts scalability to high-resolution images, and (ii) they employ guidance strategies with *prior*-agnostic posterior covariances, which can weaken alignment with the generative trajectory and degrade posterior coverage. In this paper, we propose **LFlow** (**L**atent Refinement via **Flow**s), a *training-free* framework for solving linear inverse problems via pretrained latent flow priors. LFlow leverages the efficiency of flow matching to perform ODE sampling in latent space along an optimal path. This latent formulation further allows us to introduce a theoretically grounded posterior covariance, derived from the optimal vector field, enabling effective flow guidance. Experimental results demonstrate that our proposed method outperforms state-of-the-art latent diffusion solvers in reconstruction quality across most tasks.