GenAI4ELab/papercli-papers huggingface dataset:
Search Self-Play: Pushing the Frontier of Agent Capability without Supervision
Authors: Hongliang Lu; Yuhang Wen; Pengyu Cheng; Ruijin Ding; Jiaqi Guo; Haotian Xu; Chutian Wang; Haonan Chen; xiaoxi jiang; guanjunjiang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become the mainstream technique for training LLM agents. However, RLVR highly depends on well-crafted task queries and corresponding ground-truth answers to provide accurate rewards, which requires significant human effort and hinders the scaling of RL processes, especially in agentic scenarios. Although a few recent works explore task synthesis methods, the difficulty of generated agentic tasks can hardly be controlled to provide effective RL training advantages. To achieve agentic RLVR with higher scalability, we explore self-play training for deep search agents, in which the learning LLM utilizes multi-turn search engine calling and acts simultaneously as both a task proposer and a problem solver. The task proposer aims to generate deep search queries with well-defined ground-truth answers and increasing task difficulty. The problem solver tries to handle the generated search queries and output the correct answer predictions. To ensure that each generated search query has accurate ground truth, we collect all the searching results from the proposer's trajectory as external knowledge, then conduct retrieval-augmentation generation (RAG) to test whether the proposed query can be correctly answered with all necessary search documents provided. In this search self-play (SSP) game, the proposer and the solver co-evolve their agent capabilities through both competition and cooperation. With substantial experimental results, we find that SSP can significantly improve search agents' performance uniformly on various benchmarks without any supervision under both from-scratch and continuous RL training setups. The code is at https://github.com/Qwen-Applications/SSP.
ArtVIP: Articulated Digital Assets of Visual Realism, Modular Interaction, and Physical Fidelity for Robot Learning
Authors: Zhao Jin; Zhengping Che; Tao Li; Zhen Zhao; Kun Wu; Yuheng Zhang; Yinuo Zhao; Zehui Liu; Qiang Zhang; Xiaozhu Ju; Jing Tian; Yousong Xue; Jian Tang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Robot learning increasingly relies on simulation to advance complex abilities such as dexterous manipulation and precise interaction, necessitating high-quality digital assets to bridge the sim-to-real gap. However, existing open-source articulated-object datasets for simulation are limited by insufficient visual realism and low physical fidelity, which hinders their utility for training models to master robotic tasks in the real world. To address these challenges, we introduce ArtVIP, a comprehensive open-source dataset comprising high-quality digital-twin articulated objects, accompanied by indoor-scene assets. Crafted by professional 3D modelers adhering to unified standards, ArtVIP ensures visual realism through precise geometric meshes and high-resolution textures, while physical fidelity is achieved via fine-tuned dynamic parameters. Meanwhile, the dataset pioneers embedded modular interaction behaviors within assets and pixel-level affordance annotations. Feature-map visualization and optical motion capture are employed to quantitatively demonstrate ArtVIP’s visual and physical fidelity, and its applicability is validated through imitation learning and reinforcement learning experiments. Provided in USD format with detailed production guidelines, ArtVIP is fully open-source, benefiting the research community and advancing robot learning research.
An Improved Model-free Decision-estimation Coefficient with Applications in Adversarial MDPs
Authors: Haolin Liu; Chen-Yu Wei; Julian Zimmert
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We study decision making with structured observation (DMSO). The complexity
for DMSO has been characterized by a series of work [ FKQR21 , CMB22 , FGH23 ].
Still, there is a gap between known regret upper and lower bounds: current upper
bounds incur a model estimation error that scales with the size of the model class.
The work of [FGQ+23 ] made an initial attempt to reduce the estimation error to
only scale with the size of the value function set, resulting in the complexity called
optimistic decision-estimation coefficient (optimistic DEC). Yet, their approach
relies on the optimism principle to drive exploration, which deviates from the
general idea of DEC that drives exploration only through information gain.

In this work, we introduce an improved model-free DEC, called Dig-DEC, that
removes the optimism mechanism in [FGQ+23 ], making it more aligned with
existing model-based DEC. Dig-DEC is always upper bounded by optimistic DEC,
and could be significantly smaller in special cases. Importantly, the removal of
optimism allows it to seamlessly handle adversarial environments, while it was
unclear how to achieve it within the optimistic DEC framework. By applying
Dig-DEC to hybrid MDPs where the transition is stochastic but the reward is
adversarial, we provide the first model-free regret bounds in hybrid MDPs with
bandit feedback in multiple settings: bilinear classes, Bellman-complete MDPs
with bounded Bellman-eluder dimension or coverability, resolving the main open
problem left by [LWZ25].

We also improve online function-estimation procedure used in model-free learning:
For average estimation error minimization, we improve the estimator to achieve
better concentration. This improves the  $T^{\frac{3}{4}}$ and  $T^{\frac{5}{6}}$ regret of [FGQ+23 ] to $T^{\frac{2}{3}}$and
 $T^{\frac{7}{9}}$ in the cases with on-policy and off-policy exploration. For squared estimation
error minimization in Bellman-complete MDPs, we redesign the two-timescale
procedure in [ AZ22 , FGQ+23], achieving $\sqrt{T}$ regret that improves over the  $T^{\frac{2}{3}}$
regret by [ FGQ+23 ]. This is the first time the performance of a DEC-based
approach for Bellman-complete MDPs matches that of optimism-based approaches
[JLM21, XFB+23].
GraphShield: Graph-Theoretic Modeling of Network-Level Dynamics for Robust Jailbreak Detection
Authors: Sunghee Dong; Sungwon Yi; Kangmin Bae; Jaeyoon Kim; Seongyeop Kim
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large language models (LLMs) are increasingly deployed in real-world applications but remain highly vulnerable to jailbreak prompts that bypass safety guardrails and elicit harmful outputs. We propose GraphShield, a graph-theoretic jailbreak detector that models information routing inside the LLM as token--layer graphs. Unlike prior defenses that rely on surface cues or costly gradient signals, GraphShield captures network-level dynamics in a lightweight and model-agnostic way by extracting multi-scale structural and semantic features that reveal jailbreak signatures. Extensive experiments on LLaMA-2-7B-Chat and Vicuna-7B-v1.5 show that GraphShield reduces attack success rates to 1.9% and 7.8%, respectively, while keeping refusal rates on benign prompts at 7.1% and 6.8%, significantly improving the robustness–utility trade-off compared to strong baselines. These results demonstrate that graph-theoretic modeling of network-level dynamics provides a principled and effective framework for robust jailbreak detection in LLMs.
An Agentic Framework with LLMs for Solving Complex Vehicle Routing Problems
Authors: Ni Zhang; Zhiguang Cao; Jianan Zhou; Cong Zhang; Yew-Soon Ong
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Complex vehicle routing problems (VRPs) remain a fundamental challenge, demanding substantial expert effort for intent interpretation and algorithm design. While large language models (LLMs) offer a promising path toward automation, current approaches still rely on external intervention, which restrict autonomy and often lead to execution errors and low solution feasibility. To address these challenges, we propose an Agentic Framework with LLMs (AFL) for solving complex vehicle routing problems, achieving full automation from problem instance to solution. AFL directly extracts knowledge from raw inputs and enables self-contained code generation without handcrafted modules or external solvers. To improve trustworthiness, AFL decomposes the overall pipeline into three manageable subtasks and employs four specialized agents whose coordinated interactions enforce cross-functional consistency and logical soundness. Extensive experiments on 60 complex VRPs, ranging from standard benchmarks to practical variants, validate the effectiveness and generality of our framework, showing comparable performance against meticulously designed algorithms. Notably, it substantially outperforms existing LLM-based baselines in both code reliability and solution feasibility, achieving rates close to 100% on the evaluated benchmarks.
Beyond Length: Quantifying Long-Range Information for Long-Context LLM Pretraining Data
Authors: Haoran Deng; Yingyu Lin; Zhenghao Lin; Xiao Liu; Yizhou Sun; Yian Ma; Yeyun Gong
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Long-context language models unlock advanced capabilities in reasoning, code generation, and document summarization by leveraging dependencies across extended spans of text. However, a significant portion of readily available long-text data lacks meaningful long-distance dependencies; most spans can be predicted using only local context. Training on such data is inefficient, making careful data selection crucial. Therefore, we introduce LongFilter, a framework for curating training data tailored to long-context pretraining. LongFilter measures the information gain provided by extended context by contrasting model predictions under long-context versus short-context settings, thereby identifying samples where long-range dependencies are essential. Experiments with LLaMA-3-8B, extending its context length from 8K to 64K, show that LongFilter efficiently selects high-quality data and yields substantial improvements on benchmarks such as HELMET, LongBench, and RULER.
Compositional Visual Planning via Inference-Time Diffusion Scaling
Authors: Yixin Zhang; Yunhao Luo; Utkarsh Aashu Mishra; Woo Chul Shin; Yongxin Chen; Danfei Xu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Diffusion models excel at short-horizon robot planning, yet scaling them to long-horizon tasks remains challenging due to computational constraints and limited training data. 
Existing compositional approaches stitch together short segments by separately denoising each component and averaging overlapping regions. However, this suffers from instability as the factorization assumption breaks down in noisy data space, leading to inconsistent global plans. We propose that the key to stable compositional generation lies in enforcing boundary agreement on the estimated clean data (Tweedie estimates) rather than on noisy intermediate states. Our method formulates long-horizon planning as inference over a chain-structured factor graph of overlapping video chunks, where pretrained short-horizon video diffusion models provide local priors. At inference time, we enforce boundary agreement through a novel combination of synchronous and asynchronous message passing that operates on Tweedie estimates, producing globally consistent guidance without requiring additional training. Our training-free framework demonstrates significant improvements over existing baselines, effectively generalizing to unseen start-goal combinations that were not present in the original training data. Project website: https://comp-visual-planning.github.io/
TEN-DM: Topology-Enhanced Diffusion Model for Spatio-Temporal Event Prediction
Authors: Yuxin Liu; Kaiming Wang; Chenguang Yang; Yulia Gel; Yuzhou Chen
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Spatio-temporal point process (STPP) data appear in many domains. A natural way to model them is to describe how the instantaneous event rate varies over space and time given the observed history which enables interpretation, interaction detection, and forecasting. Traditional parametric kernel-based models, while historically dominant, struggle to capture complex nonlinear patterns. In contrast, deep learning methods leverage the representational power of neural networks to aggregate historical events and integrate spatio-temporal point processes. However, existing deep learning methods often process space and time independently, overlooking the spatio-temporal dependencies. To address this limitation, we propose a novel method called Topology-ENhanced Diffusion Model (TEN-DM), including two key components namely spatio-temporal graph construction and multimodal topological feature representation learning. Further, we use temporal query technique to effectively capture periodic temporal patterns for learning effective temporal representations. Extensive experiments show the effectiveness of TEN-DM on multiple STPP datasets compared to state-of-the-art methods.
Task-Related Token Compression in Multimodal Large Language Models from an Explainability Perspective
Authors: Lei Lei; Jie Gu; Xiaokang Ma; Chu Tang; Jingmin Chen; Tong Xu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Existing Multimodal Large Language Models (MLLMs) process a large number of visual tokens, leading to significant computational costs and inefficiency. Instruction-related visual token compression demonstrates strong task relevance, which aligns well with MLLMs’ ultimate goal of instruction following. Previous works generally assume that visual tokens achieve better vision–language alignment in the shallow layers of LLMs, which have led to task-related token compression being primarily applied in intermediate LLM layers. In contrast, our study reveals that with proper selection, task-related token compression is feasible at the input stage of LLM with negligible performance loss. This new paradigm significantly reduces task-irrelevant visual tokens and its model-agnostic design enables application without modifying the LLM architecture. Specifically, we suggest that explainability methods for transformer-based architechtures can evaluate the global importance of each visual token with respect to the given instruction, which can effectively guide the task-related token compression for MLLMs. Furthermore, we propose to learn a mapping from the attention map of the first LLM layer to the explanation results, thereby avoiding the need for a full inference pass. Interestingly, this mapping can be learned using a simple and lightweight convolutional network, whose training is efficient and independent of MLLMs. Extensive experiments on 13 image and video benchmarks across three leading MLLMs (Qwen2-VL, LLaVA-OneVision, and VILA1.5) demonstrate the remarkable effectiveness and strong generalization of our approach. Additionally, our new compression paradigm achieves faster inference with reductions in both prefilling time and KV-cache memory.
Rényi Sharpness: A Novel Sharpness that Strongly Correlates with Generalization
Authors: Qiaozhe Zhang; Jun Sun; Yingzhuang Liu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Sharpness (of the loss minima) is widely believed to be a good indicator of generalization of neural networks. Unfortunately, the correlation between existing sharpness measures and the generalization is not that strong as expected, sometimes even contradiction occurs. To address this problem, a key observation in this paper is: what really matters for the generalization is the *average spread* (or unevenness) of the spectrum of loss Hessian $\mathbf{H}$. For this reason, the conventional sharpness measures, such as the trace sharpness $\operatorname{tr}(\mathbf{H})$, which cares about the *average value* of the spectrum,  or the max-eigenvalue sharpness $\lambda_{\max}(\mathbf{H})$), which concerns the  *maximum spread* of the spectrum, are not sufficient to well predict the generalization. To finely characterize the average spread of the Hessian spectrum, we leverage the notion of *Rényi entropy* in information theory, which is capable of capturing the unevenness of a probability vector and thus can be extended to describe the unevenness for a general non-negative vector (which is the case for the Hessian spectrum at the loss minima). In specific, in this paper we propose the *Rényi sharpness*, which is defined as the negative of the Rényi entropy of loss Hessian $\mathbf{H}$. 
Extensive experiments demonstrate that Rényi sharpness exhibit *strong* and *consistent* correlation with generalization in various scenarios. Moreover, on the theoretical side, two generalization bounds with respect to the Rényi sharpness are  established, by exploiting the desirable reparametrization invariance property of Rényi sharpness. Finally, as an initial attempt to take advantage of the  Rényi sharpness for regularization, Rényi Sharpness Aware Minimization (RSAM) algorithm is proposed where a variant of Rényi Sharpness is used as the regularizer. It turns out this RSAM is competitive with the state-of-the-art SAM algorithms, and far better than the conventional SAM algorithm based on the max-eigenvalue sharpness.
DEAS: DEtached value learning with Action Sequence for Scalable Offline RL
Authors: Changyeon Kim; Haeone Lee; Younggyo Seo; Kimin Lee; Yuke Zhu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Offline reinforcement learning (RL) presents an attractive paradigm for training intelligent agents without expensive online interactions. However, current approaches still struggle with complex, long-horizon sequential decision making. In this work, we introduce DEtached value learning with Action Sequence (DEAS), a simple yet effective offline RL framework that leverages action sequences for value learning. These temporally extended actions provide richer information than single-step actions, enabling reduction of the effective planning horizon by considering longer sequences at once. However, directly adopting such sequences in actor-critic algorithms introduces excessive value overestimation, which we address through detached value learning that steers value estimates toward in-distribution actions that achieve high returns in the offline dataset. We demonstrate that DEAS consistently outperforms baselines on complex, long-horizon tasks from OGBench and can be applied to enhance the performance of large-scale Vision-Language-Action models that predict action sequences, significantly boosting performance in both RoboCasa Kitchen simulation tasks and real-world manipulation tasks.
FlowRL: Matching Reward Distributions for LLM Reasoning
Authors: Xuekai Zhu; Daixuan Cheng; Dinghuai Zhang; Hengli Li; Kaiyan Zhang; Che Jiang; Youbang Sun; Ermo Hua; Yuxin Zuo; Xingtai Lv; Qizheng Zhang; Lin Chen; Fanghao Shao; Bo Xue; Yunchong Song; Zhenjie Yang; Ganqu Cui; Ning Ding; Jianfeng Gao; Xiaodong Liu; Bowen Zhou; Hongyuan Mei; Zhouhan Lin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We propose FlowRL: matching the full reward distribution via flow balancing instead of solely maximizing rewards in large language model (LLM) reinforcement learning (RL). Recent advanced reasoning models adopt reward-maximizing methods (e.g., PPO and GRPO), which tend to over-optimize dominant reward signals while neglecting less frequent but valid reasoning paths, thus reducing diversity. In contrast, we transform scalar rewards into a normalized target distribution using a learnable partition function, and then minimize the reverse KL divergence between the policy and the target distribution. We implement this idea as a flow-balanced optimization method that promotes diverse exploration and generalizable reasoning trajectories. We conduct experiments on both math and code reasoning tasks: FlowRL achieves a significant average improvement of $10.0\%$ over GRPO and $5.1\%$ over PPO on math benchmarks, and performs consistently better on code reasoning tasks. These results highlight reward distribution-matching as a key step toward efficient exploration and diverse reasoning in LLM reinforcement learning.
A Problem-Oriented Perspective and Anchor Verification for Code Optimization
Authors: Tong Ye; Tengfei Ma; Xuhong Zhang; Hang Yu; Jianwei Yin; Wenhai Wang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in solving various programming tasks, such as code generation. However, their potential for code optimization, particularly in performance enhancement, remains largely unexplored. This paper investigates the capabilities of LLMs in optimizing code for minimal execution time, addressing a critical gap in current research. The recently proposed code optimization methods constructs program optimization pairs based on iterative submissions from the same programmer for the same problem. However, this approach confines LLMs to local performance improvements, neglecting global algorithmic innovation. To overcome this limitation, we adopt a completely different perspective by reconstructing the optimization pairs into a problem-oriented approach. This allows for the integration of various ideas from multiple programmers tackling the same problem. Furthermore, we observe that code optimization presents greater challenges compared to code generation, often accompanied by "optimization tax". Recognizing the inherent trade-offs in correctness and efficiency, we introduce a novel anchor verification framework to mitigate this "optimization tax." Ultimately, the problem oriented perspective combined with anchor verification framework significantly enhances both the correct optimization ratio and speedup to new levels.
In-the-Flow Agentic System Optimization for Effective Planning and Tool Use
Authors: Zhuofeng Li; Haoxiang Zhang; Seungju Han; Sheng Liu; Jianwen Xie; Yu Zhang; Yejin Choi; James Zou; Pan Lu
Venue: ICLR 2026 (ICLR 2026 Oral)

Abstract: Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios. Agentic systems offer a promising alternative by decomposing work across specialized modules, yet most remain training-free or rely on offline training decoupled from the live dynamics of multi-turn interaction. We introduce AgentFlow, a trainable, *in-the-flow* agentic framework that coordinates four modules (planner, executor, verifier, generator) through an evolving memory and directly optimizes its planner inside the multi-turn loop. To train on-policy in live environments, we propose *Flow-based Group Refined Policy Optimization* (Flow-GRPO), which tackles long-horizon, sparse-reward credit assignment by converting multi-turn optimization into a sequence of tractable single-turn policy updates. It broadcasts a single, verifiable trajectory-level outcome to every turn to align local planner decisions with global success and stabilizes learning with group-normalized advantages. Across ten benchmarks, AgentFlow with a 7B-scale backbone outperforms top-performing baselines with average accuracy gains of 14.9% on search, 14.0% on agentic, 14.5% on mathematical, and 4.1% on scientific tasks, even surpassing larger proprietary models like GPT-4o. Further analyses confirm the benefits of in-the-flow optimization, showing improved planning, enhanced tool-calling reliability, and positive scaling with model size and reasoning turns.