GenAI4ELab/papercli-papers huggingface dataset:
Relationship Alignment for View-aware Multi-view Clustering
Authors: Shuangmei Peng; Zhe Chen; Tianyang Xu; Xiaojun Wu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Multi-view clustering improves clustering performance by integrating complementary information from multiple views. However, existing methods often suffer from two limitations: i) the neglect of preserving sample neighborhood structures, which weakens the consistency of inter-sample relationships across views; and ii) inability to adaptively utilize inter-view similarity, resulting in representation conflicts and semantic degradation. To address these issues, we propose a novel framework named Relationship Alignment for View-aware Multi-view Clustering (RAV). Our approach first constructs view-specific sample relationship matrices from deep features and aligns them with the global relationship matrix to enhance cross-view neighborhood consistency and facilitate accurate measurement of inter-view similarity. Simultaneously, we introduce a view-aware adaptive weighting mechanism for label contrastive learning that dynamically adjusts the contrastive intensity between view pairs based on deep-feature similarity: higher-similarity views lead to stronger label alignment, while lower-similarity views reduce the weighting to prevent enforcing agreement. This strategy promotes cluster-level semantic consistency while preserving natural inter-view relationships. Extensive experiments demonstrate that our method consistently outperforms state-of-the-art approaches on multiple benchmark datasets. Project website: https://github.com/chenzhe207/RAV.
Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
Authors: Yeongbin Seo; Gayoung Kim; Jaehyung Kim; Jinyoung Yeo
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: As large language models (LLMs) are pretrained on massive web corpora, careful selection of data becomes essential to ensure effective and efficient learning. While perplexity (PPL)-based filtering has demonstrated strong performance, it suffers from drawbacks: substantial time costs and inherent unreliability of the model when handling noisy or out-of-distribution samples. In this work, we propose a simple yet powerful alternative: a prior-based data filtering method that estimates token priors using corpus-level term frequency statistics, inspired by linguistic insights on word roles and lexical density. Our approach filters documents based on the mean and standard deviation of token priors, serving as a fast proxy to PPL while requiring no model inference. Despite its simplicity, the prior-based filter achieves the highest average performance across 20 downstream benchmarks, while reducing time cost by over 1000× compared to PPL-based filtering. We further demonstrate its applicability to symbolic languages such as code and math, and its dynamic adaptability to multilingual corpora without supervision.
RESTRAIN: From Spurious Votes to Signals — Self-Training RL with Self-Penalization
Authors: ZHAONING YU; Zhaolun Su; Leitian Tao; Haozhu Wang; Aashu Singh; Hanchao Yu; Jianyu Wang; Hongyang Gao; Weizhe Yuan; Jason E Weston; Ping Yu; Jing Xu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Reinforcement learning with human-annotated data has boosted chain-of-thought
reasoning in large reasoning models, but these gains come at high costs in labeled
data while faltering on harder tasks. A natural next step is experience-driven learning, where models improve without curated labels by adapting to unlabeled data.
We introduce REinforcement learning with Self-resTRAINt training (RESTRAIN),
a self-penalizing RL framework that converts the absence of gold labels into a
useful learning signal. Instead of overcommitting to spurious majority votes,
RESTRAIN exploits signals from the model’s entire answer distribution: penalizing
overconfident rollouts and low-consistency examples while preserving promising
reasoning chains. This self-penalization mechanism integrates seamlessly into
policy optimization methods such as GRPO, enabling continual self-improvement
without supervision. On challenging reasoning benchmarks, RESTRAIN delivers
large gains using only unlabeled data. With Qwen3-4B-Base and OctoThinker
Hybrid-8B-Base, it boosts Pass@1 by up to +140.7% on AIME25, +36.2% on
MMLU STEM, and +19.6% on GPQA-Diamond, nearly matching gold-label
training while using no gold labels. These results demonstrate that RESTRAIN establishes a scalable path toward stronger reasoning without gold labels.
Erase or Hide? Suppressing Spurious Unlearning Neurons for Robust Unlearning
Authors: Nakyeong Yang; Dong-Kyum Kim; Jea Kwon; Minsung Kim; Kyomin Jung; Meeyoung Cha
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large language models trained on web-scale data can memorize private or sensitive knowledge, raising significant privacy risks.
Although some unlearning methods mitigate these risks, they remain vulnerable to "relearning" during subsequent training, allowing a substantial portion of forgotten knowledge to resurface.
In this paper, we show that widely used unlearning methods cause shallow alignment: instead of faithfully erasing target knowledge, they generate spurious unlearning neurons that amplify negative influence to hide it.
To overcome this limitation, we introduce Ssiuu, a new class of unlearning methods that employs attribution-guided regularization to prevent spurious negative influence and faithfully remove target knowledge.
Experimental results confirm that our method reliably erases target knowledge and outperforms strong baselines across two practical retraining scenarios: (1) adversarial injection of private data, and (2) benign attack using an instruction-following benchmark.
Our findings highlight the necessity of robust and faithful unlearning methods for safe deployment of language models.
Fewer Weights, More Problems: A Practical Attack on LLM Pruning
Authors: Kazuki Egashira; Robin Staab; Thibaud Gloaguen; Mark Vero; Martin Vechev
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Model pruning, i.e., removing a subset of model weights, has become a prominent approach to reducing the memory footprint of large language models (LLMs) during inference.
Notably, popular inference engines, such as vLLM, enable users to conveniently prune downloaded models before they are deployed.
While the utility and efficiency of pruning methods have improved significantly, the security implications of pruning remain underexplored.
In this work, for the first time, we show that modern LLM pruning methods can be maliciously exploited.
In particular, an adversary can construct a model that appears benign yet, once pruned, exhibits malicious behaviors.
Our method is based on the idea that the adversary can compute a proxy metric that estimates how likely each parameter is to be pruned.
With this information, the adversary can first inject a malicious behavior into those parameters that are unlikely to be pruned.
Then, they can repair the model by using parameters that are \textit{likely} to be pruned, effectively canceling out the injected behavior in the unpruned model.
We demonstrate the severity of our attack through extensive evaluation on five models; after any of the pruning in vLLM are applied (Magnitude, Wanda, and SparseGPT), it consistently exhibits strong malicious behaviors in a diverse set of attack scenarios (success rates of up to 95.7% for jailbreak, 98.7% for benign instruction refusal, and 99.5% for targeted content injection).
Our results reveal a critical deployment-time security gap and underscore the urgent need for stronger security awareness in model compression.
Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
Authors: Shuhua Yu; Dusan Jakovetic; Soummya Kar
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Heavy-tailed noise in nonconvex stochastic optimization has garnered increasing research interest, as empirical studies, including those on training attention models, suggest it is a more realistic gradient noise condition. This paper studies first-order nonconvex stochastic optimization under heavy-tailed gradient noise in a decentralized setup, where each node can only communicate with its direct neighbors in a predefined graph. Specifically, we consider a class of heavy-tailed gradient noise that is zero-mean and has only $p$-th moment for $p \in (1, 2]$. We propose GT-NSGDm, Gradient Tracking based Normalized Stochastic Gradient Descent with momentum, that utilizes normalization, in conjunction with gradient tracking and momentum, to cope with heavy-tailed noise on distributed nodes. We show that, when the communication graph admits primitive and doubly stochastic weights, GT-NSGDm guarantees, for the first time in the literature, that the expected gradient norm converges at an optimal non-asymptotic rate $O\big(1/T^{(p-1)/(3p-2)}\big)$, which matches the lower bound in the centralized setup. When the tail index $p$ is unknown, GT-NSGDm attains a non-asymptotic rate $O\big( 1/T^{(p-1)/(2p)} \big)$ that is, for $p < 2$, topology independent and has a speedup factor $n^{1-1/p}$ in terms of the number of nodes $n$. Finally, experiments on nonconvex linear regression with tokenized synthetic data and decentralized training of language models on a real-world corpus demonstrate that GT-NSGDm is more robust and efficient than  baselines.
You Point, I Learn: Online Adaptation of Interactive Segmentation Models for Handling Distribution Shifts in Medical Imaging
Authors: Wentian Xu; Ziyun Liang; Harry Anthony; Yasin Ibrahim; Felix Cohen; Guang Yang; Konstantinos Kamnitsas
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Interactive segmentation uses real-time user inputs, such as mouse clicks, to iteratively refine model predictions. Although not originally designed to address distribution shifts, this paradigm naturally lends itself to such challenges. In medical imaging, where distribution shifts are common, interactive methods can use user inputs to guide models towards improved predictions.
Moreover, once a model is deployed, user corrections can be used to adapt the network parameters to the new data distribution, mitigating distribution shift. Based on these insights, we aim to develop a practical, effective method for improving the adaptive capabilities of interactive segmentation models to new data distributions in medical imaging.  Firstly, we found that strengthening the model's responsiveness to clicks is important for the initial training process. Moreover, we show that by treating the post-interaction user-refined model output as pseudo-ground-truth, we can design a lean, practical online adaptation method that enables a model to learn effectively across sequential test images. The framework includes two components: (i) a Post-Interaction adaptation process, updating the model after the user has completed interactive refinement of an image, and (ii) a Mid-Interaction adaptation process, updating incrementally after each %user 
click. Both processes include a Click-Centered Gaussian loss that strengthens the model's reaction to clicks and enhances focus on user-guided, clinically relevant regions.
Experiments on 5 fundus and 4 brain‑MRI databases show that our approach consistently outperforms existing methods under diverse distribution shifts, including unseen imaging modalities and pathologies.
Code: https://github.com/WenTXuL/OAIMS
Statistical Guarantees for Offline Domain Randomization
Authors: Arnaud Fickinger; Abderrahim Bendahi; Stuart Russell
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Reinforcement-learning agents often struggle when deployed from simulation to the real-world. A dominant strategy for reducing the sim-to-real gap is domain randomization (DR) which trains the policy across many simulators produced by sampling dynamics parameters, but standard DR ignores offline data already available from the real system. We study offline domain randomization (ODR), which first fits a distribution over simulator parameters to an offline dataset. While a growing body of empirical work reports substantial gains with algorithms such as DROPO, the theoretical foundations of ODR remain largely unexplored. In this work, we cast ODR as a maximum-likelihood estimation over a parametric simulator family and provide statistical guarantees: under mild regularity and identifiability conditions, the estimator is weakly consistent (it converges in probability to the true dynamics as data grows), and it becomes strongly consistent (i.e., it converges almost surely to the true dynamics) when an additional uniform Lipschitz continuity assumption holds. We examine the practicality of these assumptions and outline relaxations that justify ODR’s applicability across a broader range of settings. Taken together, our results place ODR on a principled footing and clarify when offline data can soundly guide the choice of a randomization distribution for downstream offline RL.
Decomposition of Concept-Level Rules in Visual Scenes
Authors: Fan Shi; Yuxuan Liang; Xiaolei Chen; Haiyang Yu; Xu Li; Yi Zheng; Rui Zhu; Xiangyang Xue; Bin Li
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Human cognition is compositional, and one can parse a visual scene into independent concepts and the corresponding concept-changing rules. By contrast, many vision-language systems process images holistically, with limited support for explicit decomposition. Previous methods of decomposing concepts and rules often rely on hand-crafted inductive biases or human-designed priors. We introduce a Concept-Rule Decomposition (CRD) framework to decompose concept-level rules with Large Vision-Language Models (LVLMs), which explains visual input by leveraging LVLM-extracted concepts and the rules governing their variation. The proposed method operates in two stages: (1) a pretrained LVLM proposes visual concepts and concept values, which are employed to instantiate a space of concept rule functions that model concept changes and spatial distributions; (2) an iterative process to select a concise set of concepts that best account for the input according to the rule function. We evaluate CRD on an abstract visual reasoning benchmark, a spatial reasoning benchmark, and a real-world image caption dataset. Across both settings, our approach outperforms baseline models while improving interpretability by explicitly revealing underlying concepts and compositional rules, advancing explainable and generalizable visual reasoning.
Getting Your LLMs Ready for Reinforcement Learning with Lightweight SFT
Authors: Xinran Li; Guangda Huzhang; Siqi Shen; Qing-Guo Chen; Zhao Xu; Weihua Luo; Kaifu Zhang; Jun Zhang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Reinforcement learning (RL) has emerged as a powerful post-training paradigm for large language models (LLMs), yet its effectiveness varies significantly across base models. While incorporating a pre-RL supervised fine-tuning (SFT) phase can enhance RL training, key questions remain: how long should the SFT cold-start phase last, and is the SFT objective truly aligned with the requirements for effective RL preparation?
In our analysis of cold-start dynamics, we uncover a key limitation: the SFT checkpoint with the highest evaluation performance often fails to maximize RL potential due to distributional forgetting—a phenomenon where the model drifts excessively away from the base model’s distribution even before traditional overfitting occurs. We identify diversity metrics, such as the entropy and self-BLEU, as more reliable early-stopping criteria than the standard performance-based checkpoint selection. Our findings show that SFT checkpoints with peak diversity consistently lead to superior post-RL results. Building on these insights, we introduce Adaptive Early-Stop Loss (AESL), a lightweight and dynamic cold-start method that balances the acquisition of new patterns with the preservation of the base model's distribution. AESL operates at both the token and subsequence levels, providing finer-grained control over the cold-start process. Experimental results on mathematical reasoning benchmarks demonstrate that diversity-based early stopping surpasses traditional performance-based SFT, while AESL further enhances RL preparation. By steering LLMs toward better initialization points for RL, AESL consistently achieves superior final performance compared to existing SFT and cold-start strategies. The
code is publicly available at \url{https://github.com/LXXXXR/AESL}.
How Catastrophic is Your LLM? Certifying Risks in Conversation
Authors: Chengxiao Wang; Isha Chaudhary; Qian Hu; Weitong Ruan; Rahul Gupta; Gagandeep Singh
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large Language Models (LLMs) can produce catastrophic responses in conversational settings that pose serious risks to public safety and security. Existing evaluations often fail to fully reveal these vulnerabilities because they rely on fixed attack prompt sequences, lack statistical guarantees, and do not scale to the vast space of multi-turn conversations. In this work, we propose C$^3$LLM, a novel, principled Certification framework for Catastrophic risks in multi-turn Conversation for LLMs that bounds the probability of an LLM generating catastrophic responses under multi-turn conversation distributions with statistical guarantees. We model multi-turn conversations as probability distributions over query sequences, represented by a Markov process on a query graph whose edges encode semantic similarity to capture realistic conversational flow, and quantify catastrophic risks using confidence intervals. We define several inexpensive and practical distributions—random node, graph path, and adaptive with rejection. Our results demonstrate that these distributions can reveal substantial catastrophic risks in frontier models, with certified lower bounds as high as 70\% for the worst model, highlighting the urgent need for improved safety training strategies in frontier LLMs.
Discovering alternative solutions beyond the simplicity bias in recurrent neural networks
Authors: William Qian; Cengiz Pehlevan
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Training recurrent neural networks (RNNs) to perform neuroscience-style tasks has become a popular way to generate hypotheses for how neural circuits in the brain might perform computations. Recent work has demonstrated that task-trained RNNs possess a strong simplicity bias. In particular, this inductive bias often causes RNNs trained on the same task to collapse on effectively the same solution, typically comprised of fixed-point attractors or other low-dimensional dynamical motifs. While such solutions are readily interpretable, this collapse proves counterproductive for the sake of generating a set of genuinely unique hypotheses for how neural computations might be performed. Here we propose Iterative Neural Similarity Deflation (INSD), a simple method to break this inductive bias. By penalizing linear predictivity of neural activity produced by standard task-trained RNNs, we find an alternative class of solutions to classic neuroscience-style RNN tasks. These solutions appear distinct across a battery of analysis techniques, including representational similarity metrics, dynamical systems analysis, and the linear decodability of task-relevant variables. Moreover, these alternative solutions can sometimes achieve superior performance in difficult or out-of-distribution task regimes. Our findings underscore the importance of moving beyond the simplicity bias to uncover richer and more varied models of neural computation.
A Single Architecture for Representing Invariance Under Any Space Group
Authors: Cindy Zhang; Elif Ertekin; Peter Orbanz; Ryan P Adams
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Incorporating known symmetries in data into machine learning models has consistently improved predictive accuracy, robustness, and generalization. However, achieving exact invariance to specific symmetries typically requires designing bespoke architectures for each group of symmetries, limiting scalability and preventing knowledge transfer across related symmetries.  In the case of the space groups—symmetries critical to modeling crystalline solids in materials science and condensed matter physics—this challenge is particularly salient as there are 230 such groups in three dimensions. In this work we present a new approach to crystallographic symmetries by developing a single machine learning architecture that is capable of adapting its weights automatically to enforce invariance to any input space group. Our approach is based on constructing symmetry-adapted Fourier bases through an explicit characterization of constraints that group operations impose on Fourier coefficients. Encoding these constraints into a neural network layer enables weight sharing across different space groups, allowing the model to leverage structural similarities between groups and overcome data sparsity when limited measurements are available for specific groups. We demonstrate the effectiveness of this approach in achieving competitive performance on material property prediction tasks and performing zero-shot learning to generalize to unseen groups.
Provably Explaining Neural Additive Models
Authors: Shahaf Bassan; Yizhak Yisrael Elboher; Tobias Ladner; Volkan Şahin; Jan Kretinsky; Matthias Althoff; Guy Katz
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Despite significant progress in post-hoc explanation methods for neural
networks, many remain heuristic and lack provable guarantees. A key approach
for obtaining explanations with provable guarantees is by identifying a
cardinally-minimal subset of input features which by itself is provably
sufficient to determine the prediction. However, for standard neural networks,
this task is often computationally infeasible, as it demands a worst-case
exponential number of verification queries in the number of input features,
each of which is NP-hard.
  In this work, we show that for Neural Additive Models (NAMs), a recent and
more interpretable neural network family, we can efficiently generate
explanations with such guarantees. We present a new model-specific algorithm
for NAMs that generates provably cardinally-minimal explanations using only a
logarithmic number of verification queries
  in the number of input features, after a parallelized preprocessing step with
logarithmic runtime in the required precision is applied to each small
univariate NAM component.
  Our algorithm not only makes the task of obtaining cardinally-minimal
explanations feasible, but even outperforms existing algorithms designed to
find the relaxed variant of subset-minimal explanations - which may be larger
and less informative but easier to compute - despite our algorithm solving a
much more difficult task.
  Our experiments demonstrate that, compared to previous algorithms, our
approach provides provably smaller explanations than existing works and
substantially reduces the computation time. Moreover, we show that our
generated provable explanations offer benefits that are unattainable by
standard sampling-based techniques typically used to interpret NAMs.