GenAI4ELab/papercli-papers huggingface dataset:
EvA: Evolutionary Attacks on Graphs
Authors: Mohammad Sadegh Akhondzadeh; Soroush H. Zargarbashi; Jimin Cao; Aleksandar Bojchevski
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Even a slight perturbation in the graph structure can cause a significant drop in the accuracy of graph neural networks (GNNs). Most existing attacks leverage gradient information to perturb edges. This relaxes the attack's optimization problem from a discrete to a continuous space, resulting in solutions far from optimal. It also prevents the adaptability of the attack to non-differentiable objectives. Instead, we introduce a few simple, yet effective, enhancements of an evolutionary-based algorithm to solve the discrete optimization problem directly. Our Evolutionary Attack EvA works with any black-box model and objective, eliminating the need for a differentiable proxy loss. This allows us to design two novel attacks that reduce the effectiveness of robustness certificates and break conformal sets.
EvA uses sparse representations to significantly reduce memory requirements and scale to larger graphs.
We also introduce a divide and conquer strategy that improves both EvA and existing gradient-based attacks.
Among our experiments, EvA shows $\sim$11\% additional drop in accuracy on average compared to the best previous attack, 
revealing significant untapped potential in designing attacks.
Webscale-RL: Automated Data Pipeline for Scaling RL Data to Pretraining Levels
Authors: Zhepeng Cen; Haolin Chen; Shiyu Wang; Zuxin Liu; Zhiwei Liu; Ding Zhao; Caiming Xiong; Huan Wang; Weiran Yao
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large Language Models (LLMs) have achieved remarkable success through imitation learning on vast text corpora, but this paradigm creates a training-generation gap and limits robust reasoning. Reinforcement learning (RL) offers a more data-efficient solution capable of bridging this gap, yet its application has been constrained by a critical data bottleneck: existing RL datasets are orders of magnitude smaller and less diverse than web-scale pre-training corpora. To address this, we introduce the \textbf{\texttt{Webscale-RL} pipeline}, a scalable data engine that systematically converts large-scale pre-training documents into millions of diverse, verifiable question-answer pairs for RL. Using this pipeline, we construct the \textbf{\texttt{Webscale-RL} dataset}, containing 1.2 million examples across more than 9 domains. Our experiments show that the model trained on this dataset significantly outperforms continual pretraining and strong data refinement baselines across a suite of benchmarks. Notably, RL training with our dataset proves substantially more efficient, achieving the performance of continual pre-training with up to 100$\times$ fewer tokens. Our work presents a viable path toward scaling RL to pre-training levels, enabling more capable and efficient language models.
Circuit Insights: Towards Interpretability Beyond Activations
Authors: Elena Golimblevskaia; Aakriti Jain; Bruno Puri; Ammar Ibrahim; Wojciech Samek; Sebastian Lapuschkin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The fields of explainable AI and mechanistic interpretability aim to uncover the internal structure of neural networks, with circuit discovery as a central tool for understanding model computations. Existing approaches, however, rely on manual inspection and remain limited to toy tasks. Automated interpretability offers scalability by analyzing isolated features and their activations, but it often misses interactions between features and depends strongly on external LLMs and dataset quality. Transcoders have recently made it possible to separate feature attributions into input-dependent and input-invariant components, providing a foundation for more systematic circuit analysis. Building on this, we propose WeightLens and CircuitLens, two complementary methods that go beyond activation-based analysis. WeightLens interprets features directly from their learned weights, removing the need for explainer models or datasets while matching or exceeding the performance of existing methods on context-independent features. CircuitLens captures how feature activations arise from interactions between components, revealing circuit-level dynamics that activation-only approaches cannot identify. Together, these methods increase interpretability robustness and enhance scalable mechanistic analysis of circuits while maintaining efficiency and quality.
Trust-Region Adaptive Policy Optimization
Authors: Mingyu Su; Jian Guan; Yuxian Gu; Minlie Huang; Hongning Wang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Post-training methods, especially Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), play an important role in improving large language models' (LLMs) complex reasoning abilities. However, the dominant two-stage pipeline (SFT then RL) suffers from a key inconsistency: SFT enforces rigid imitation that suppresses exploration and induces forgetting, limiting RL's potential for improvements. 
We address this inefficiency with TRAPO  (**T**rust-**R**egion **A**daptive **P**olicy **O**ptimization), a hybrid framework that interleaves SFT and RL within each training instance by optimizing SFT loss on expert prefixes and RL loss on the model's own completions, unifying external supervision and self-exploration. To stabilize training, we introduce Trust-Region SFT (TrSFT), which minimizes forward KL divergence inside a trust region but attenuates optimization outside, effectively shifting toward reverse KL and yielding stable, mode-seeking updates favorable for RL. An adaptive prefix-selection mechanism further allocates expert guidance based on measured utility. Experiments on five mathematical reasoning benchmarks show that TRAPO consistently surpasses standard SFT, RL, and SFT-then-RL pipelines, as well as recent state-of-the-art approaches, establishing a strong new paradigm for reasoning-enhanced LLMs.
Decoupling Positional and Symbolic Attention in Transformers
Authors: Felipe Urrutia; Jorge Salas; Alexander Kozachinskiy; Cristian Buc Calderon; Hector Pasten; Cristobal Rojas
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: An important aspect subtending language understanding and production is the ability to independently encode positional and symbolic information of the words within a sentence. In Transformers, positional information is typically encoded using Positional Encodings (PEs). One such popular PE, namely Rotary PE (RoPE), has been widely used due to its empirical success. 
Recently, it has been argued that part of RoPE's success emerges from its ability to encode robust positional and semantic information using large and small frequencies, respectively. In this work, we perform a deeper dive into the positional versus symbolic dichotomy of attention heads behavior, both at the theoretical and empirical level. We provide general  definitions of what it means for a head to behave positionally or symbolically, prove that these are two mutually exclusive behaviors and develop a metric to quantify them.  
We apply our framework to analyze Transformer-based LLMs using RoPE and find that all heads exhibit a strong correspondence between behavior and frequency use.  
Finally, we introduce canonical tasks designed to be either purely positional or symbolic, and demonstrate that the Transformer performance causally relates to the ability of attention heads to leverage the appropriate frequencies. In particular, we show that we can control the Transformer performance by controlling which frequencies the attention heads can access. Altogether, our work provides a detailed understanding of RoPE, and how its properties relate to model behavior.
A Recovery Guarantee for Sparse Neural Networks
Authors: Sara Fridovich-Keil; Mert Pilanci
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We prove the first guarantees of sparse recovery for ReLU neural networks, where the sparse network weights constitute the signal to be recovered. Specifically, we study structural properties of the sparse network weights for two-layer, scalar-output networks under which a simple iterative hard thresholding algorithm recovers these weights exactly, using memory that grows linearly in the number of nonzero weights. We validate this theoretical result with simple experiments on recovery of sparse planted MLPs, MNIST classification, and implicit neural representations. Experimentally, we find performance that is competitive with, and often exceeds, a high-performing but memory-inefficient baseline based on iterative magnitude pruning. Code is available at https://github.com/voilalab/MLP-IHT.
An Information-Theoretic Framework For Optimizing Experimental Design To Distinguish Probabilistic Neural Codes
Authors: Po-Chen Kuo; Edgar Y. Walker
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The Bayesian brain hypothesis has been a leading theory in understanding perceptual decision-making under uncertainty.
While extensive psychophysical evidence supports the notion of the brain performing Bayesian computations, how uncertainty information is encoded in sensory neural populations remains elusive.
Specifically, two competing hypotheses propose that early sensory populations encode either the likelihood function (exemplified by probabilistic population codes) or the posterior distribution (exemplified by neural sampling codes) over the stimulus, with the key distinction lying in whether stimulus priors would modulate the neural responses.
However, experimentally differentiating these two hypotheses has remained challenging, as it is unclear what task design would effectively distinguish the two. 
In this work, we present an information-theoretic framework for optimizing the task stimulus distribution that would maximally differentiate competing probabilistic neural codes.
To quantify how distinguishable the two probabilistic coding hypotheses are under a given task design, we derive the \textit{information gap}---the expected performance difference when likelihood versus posterior decoders are applied to neural populations---by evaluating the Kullback–Leibler divergence between the true posterior and a task-marginalized surrogate posterior.
Through extensive simulations, we demonstrate that the information gap accurately predicts decoder performance differences across diverse task settings.
Critically, maximizing the information gap yields stimulus distributions that optimally differentiate likelihood and posterior coding hypotheses. 
Our framework enables principled, theory-driven experimental designs with maximal discriminative power to differentiate probabilistic neural codes, advancing our understanding of how neural populations represent and process sensory uncertainty.
CitySeeker: How Do VLMs Explore Embodied Urban Navigation with Implicit Human Needs?
Authors: Siqi Wang; Chao Liang; Yunfan Gao; Erxin Yu; Sen Li; Jing Li; Haofen Wang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Vision-Language Models (VLMs) have made significant progress in explicit instruction-based navigation; however, their ability to interpret implicit human needs (e.g., ''I am thirsty'') in dynamic urban environments remains underexplored. This paper introduces CitySeeker, a novel benchmark designed to assess VLMs’ spatial reasoning and decision-making capabilities for exploring embodied urban navigation to address implicit needs. CitySeeker includes 6,440 trajectories across 8 cities, capturing diverse visual characteristics and implicit needs in 7 goal-driven scenarios. Extensive experiments reveal that even top-performing models (e.g., Qwen2.5-VL-32B-Instruct) achieve only 21.1% task completion. We find key bottlenecks in error accumulation in long-horizon reasoning, inadequate spatial cognition, and deficient experiential recall. To further analyze them, we investigate a series of exploratory strategies—Backtracking Mechanisms, Enriching Spatial Cognition, and Memory-Based Retrieval (BCR), inspired by human cognitive mapping's emphasis on iterative observation-reasoning cycles and adaptive path optimization. Our analysis provides actionable insights for developing VLMs with robust spatial intelligence required for tackling ''last-mile'' navigation challenges.
In-Context Learning for Pure Exploration
Authors: Alessio Russo; Ryan Welch; Aldo Pacchiano
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We study the _active sequential hypothesis testing_ problem, also known as _pure exploration_: given a new task, the learner _adaptively collects data_ from the environment to  efficiently determine an underlying correct hypothesis. A classical instance of this problem is the task of identifying the best arm in a multi-armed bandit problem (a.k.a. BAI, Best-Arm Identification), where actions index hypotheses. Another important case is generalized  search, a problem of determining the correct label through a sequence of strategically
selected queries that indirectly reveal information about the label.
In this work, we introduce _In-Context Pure Explorer_ (ICPE), which meta-trains Transformers to map _observation histories_ to _query actions_ and a _predicted hypothesis_, yielding a model that transfers in-context. At inference time,  ICPE actively gathers evidence on new tasks and infers the true hypothesis without parameter updates.
Across deterministic, stochastic, and structured benchmarks, including BAI  and generalized search,  ICPE is competitive with  adaptive baselines while requiring no explicit modeling of information structure. Our results support Transformers as practical architectures for _general sequential testing_.
Mode-conditioning unlocks superior test-time compute scaling
Authors: Chen Henry Wu; Sachin Goyal; Aditi Raghunathan
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Parallel sampling is essential to test-time scaling and reinforcement learning (RL), but its effectiveness is sharply limited by diversity collapse, where models concentrate on a few modes and repeated samples produce the same mistakes. We propose the mode-conditioning (ModC) framework, which explicitly allocates sampling compute across reasoning modes using either specialist models or mode-specific prefixes. With predefined mode labels, ModC consistently improves test-time scaling (Pass@k) across controlled graph-search tasks and math reasoning benchmarks, spanning model families and sizes from 0.5B to 7B. On OpenThoughts, fine-tuning Qwen2.5-7B with ModC achieves an 4× efficiency gain over standard training while also improving the maximum attainable Pass@k. We further show that gradient clustering enables ModC without predefined mode labels, yielding up to 10% gains on datasets such as NuminaMath. Finally, we show that ModC improves Pass@k after RL training and can further boost the Pass@k gains of diversity-inducing RL methods. These results demonstrate that standard training underutilizes the diversity in data, and that ModC provides a simple, effective remedy for unlocking the full benefits of diversity in parallel sampling.
HARDTESTGEN: A High-Quality RL Verifier Generation Pipeline for LLM Algorithmic Coding
Authors: Zhongmou He; Yee Man Choi; Kexun Zhang; Ivan Bercovich; Jiabao Ji; Junting Zhou; Dejia Xu; Aidan Zhang; Yixiao Zeng; Lei Li
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Verifiers provide important reward signals for reinforcement learning of large language models (LLMs).
However, it is challenging to develop or create reliable verifiers, especially for code generation tasks.
A well-disguised wrong solution program may only be detected by carefully human-written edge cases that are difficult to synthesize automatically. To address this issue, we propose HARDTESTGEN, an approach to synthesize high-quality test cases for algorithmic coding problems. We curate a comprehensive algorithmic programming dataset HARDTESTS with 26.6k problems and high-quality synthetic tests.
Compared with existing tests, \method tests demonstrate significantly higher accuracy in verifying LLM-generated code (+11.22 percentage points in precision, the percentage of actually correct code within the predicted correct ones). We also show that downstream post-training --- including rejection sampling and reinforcement learning (RL) --- using HARDTESTS verifier results in improved performance of LLM code generation.  We open-source our dataset and synthesis pipeline at https://leililab.github.io/HardTests/.
OpenApps: Simulating Environment Variations to Measure UI Agent Reliability
Authors: Karen Ullrich; Jingtong Su; Claudia Shi; Arjun Subramonian; Amir Bar; Ivan Evtimov; Nikolaos Tsilivis; Randall Balestriero; Julia Kempe; Mark Ibrahim
Venue: ICLR 2026 (ICLR 2026 Oral)

Abstract: Reliability is key to realizing the promise of autonomous UI-agents, multimodal agents that directly interact with the apps humans use, as users must be able to trust an agent to complete a given task. Current evaluations rely on fixed environments---often clones of existing apps--- which are limited in that they can only shed light on whether or how often an agent can complete a task within a specific environment. When deployed however, agents are likely to encounter variations in app design and content that can affect an agent’s ability to complete a task. To address this blind spot of measuring agent reliability across app variations, we develop OpenApps, a light-weight open-source ecosystem with six apps (messenger, calendar, maps, etc.) that are configurable in appearance and content. OpenApps requires just a single CPU to run, enabling easy generation and deployment of thousands of versions of each app. Specifically, we run more than 10,000 independent evaluations to study reliability across seven leading multimodal agents. We find that while standard reliability within a fixed app is relatively stable, reliability can vary drastically when measured across app variations. Task success rates for many agents can fluctuate by more than 50\% across app variations. For example, Kimi-VL-3B's average success across all tasks fluctuates from 63\% to just 4\% across app versions. We also find agent behaviors such as looping or hallucinating actions can differ drastically depending on the environment configuration. These initial findings highlight the importance of measuring reliability along this new dimension of app variations.
Discrete Compositional Generation via General Soft Operators and Robust Reinforcement Learning
Authors: Marco Jiralerspong; Esther Derman; Danilo Vucetic; Nikolay Malkin; Bilun Sun; Tianyu Zhang; Pierre-Luc Bacon; Gauthier Gidel
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: A major bottleneck in scientific discovery involves narrowing an exponentially large set of objects, such as proteins or molecules, to a small set of promising candidates with desirable properties. While this process can rely on expert knowledge, recent methods leverage reinforcement learning (RL) guided by a proxy reward function to enable this filtering. By employing various forms of entropy regularization, these methods aim to learn samplers that generate diverse candidates that are highly rated by the proxy function. In this work, we make two main contributions. First, we show that these methods are liable to generate overly diverse, suboptimal candidates in large search spaces. To address this issue, we introduce a novel unified operator that combines several regularized RL operators into a general framework that better targets peakier sampling distributions. Secondly, we offer a novel, robust RL perspective of this filtering process. The regularization can be interpreted as robustness to a compositional form of uncertainty in the proxy function (i.e., the true evaluation of a candidate differs from the proxy's evaluation). Our analysis leads us to a novel, easy-to-use algorithm we name trajectory general mellowmax (TGM): we show it identifies higher quality, diverse candidates than baselines in both synthetic and real-world tasks.
Neologism Learning for Controllability and Self-Verbalization
Authors: John Hewitt; Oyvind Tafjord; Robert Geirhos; Been Kim
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Humans invent new words when there is a rising demand for a new useful concept
(e.g., doomscrolling). We explore and validate a similar idea in our communication
with LLMs: introducing new words to better understand and control the models,
expanding on the recently introduced neologism learning. This method introduces a
new word by adding a new word embedding and training with examples that exhibit
the concept with no other changes in model parameters. We show that adding a new
word allows for control of concepts such as flattery, incorrect answers, text length,
as well as more complex concepts in AxBench. We discover that neologisms can
also further our understanding of the model via self-verbalization: models can
describe what each new word means to them in natural language, like explaining
that a word that represents a concept of incorrect answers means “a lack of complete,
coherent, or meaningful answers. . . ” To validate self-verbalizations, we introduce
plug-in evaluation: we insert the verbalization into the context of a model and
measure whether it controls the target concept. In some self-verbalizations, we find
machine-only synonyms: words that seem unrelated to humans but cause similar
behavior in machines. Finally, we show how neologism learning can jointly learn
multiple concepts in multiple words.
Learning Concept Bottleneck Models from Mechanistic Explanations
Authors: Antonio De Santis; Schrasing Tong; Marco Brambilla; Lalana Kagal
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Concept Bottleneck Models (CBMs) aim for ante-hoc interpretability by learning a bottleneck layer that predicts interpretable concepts before the decision. State-of-the-art approaches typically select which concepts to learn via human specification, open knowledge graphs, prompting an LLM, or using general CLIP concepts. However, concepts defined a-priori may not have sufficient predictive power for the task or even be learnable from the available data. As a result, these CBMs often significantly trail their black-box counterpart when controlling for information leakage. To address this, we introduce a novel CBM pipeline named Mechanistic CBM (M-CBM), which builds the bottleneck directly from a black-box model’s own learned concepts. These concepts are extracted via Sparse Autoencoders (SAEs) and subsequently named and annotated on a selected subset of images using a Multimodal LLM. For fair comparison and leakage control, we also introduce the Number of Contributing Concepts (NCC), a decision-level sparsity metric that extends the recently proposed NEC metric. Across diverse datasets, we show that M-CBMs consistently surpass prior CBMs at matched sparsity, while improving concept predictions and providing concise explanations. Our code is available at https://github.com/Antonio-Dee/M-CBM.