GenAI4ELab/papercli-papers huggingface dataset:
Beyond Short Steps in Frank-Wolfe Algorithms
Authors: David Martínez-Rubio; Sebastian Pokutta
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We introduce novel techniques to enhance Frank-Wolfe algorithms by leveraging function smoothness beyond traditional short steps. Our study focuses on Frank-Wolfe algorithms with step sizes that incorporate primal-dual guarantees, offering practical stopping criteria. We present a new Frank-Wolfe algorithm utilizing an optimistic framework and provide a primal-dual convergence proof. Additionally, we propose a generalized short-step strategy aimed at optimizing a computable primal-dual gap. Interestingly, this new generalized short-step strategy is also applicable to gradient descent algorithms beyond Frank-Wolfe methods. Empirical results demonstrate that our optimistic algorithm outperforms existing methods, highlighting its practical advantages.
Light Differentiable Logic Gate Networks
Authors: Lukas Rüttgers; Till Aczel; Andreas Plesner; Roger Wattenhofer
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Differentiable logic gate networks (DLGNs) exhibit extraordinary efficiency at inference while sustaining competitive accuracy.
But vanishing gradients, discretization errors, and high training cost impede scaling these networks.
Even with dedicated parameter initialization schemes from subsequent works, increasing depth still harms accuracy.
We show that the root cause of these issues lies in the underlying parametrization of logic gate neurons themselves.
To overcome this issue, we propose a reparametrization that also shrinks the parameter size logarithmically in the number of inputs per gate.
For binary inputs, this already reduces the model size by 4x, speeds up the backward pass by up to 1.86x, and converges in 8.5x fewer training steps.
On top of that, we show that the accuracy on CIFAR-100 remains stable and sometimes superior to the original parametrization.
Imitating the Truth: Attention-aware Truth-Guided Enhancement for Hallucination Mitigation in Large Vision-Language Models
Authors: Hairui Ren; Zixuan Wang; Yibo Yang; He Zhao; Fan Tang; Dandan Guo; Yi Chang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large Vision-Language Models (LVLMs) achieve impressive multimodal reasoning but remain prone to hallucinations, generating content inconsistent with visual evidence.
Existing mitigation methods often rely on auxiliary modules or coarse decoding-time adjustments, overlooking the fine-grained dynamics that distinguish truthful (real) tokens from hallucinatory ones.
In this paper, we introduce \textbf{AGE (Attention-aware Truth-Guided Enhancement)}, a training-free framework that performs fine-grained, layer-wise interventions guided by attention patterns of real tokens. 
Our analysis reveals that real and hallucinated tokens follow distinct stage-specific attention behaviors, and hallucinations emerge when models fail to reproduce these behaviors.
AGE addresses this by introducing two lightweight interventions: (i) Imitating the image attention, derived from discrepancies between real and hallucinated tokens, and (ii) Imitating the text attention when semantic grounding is required.
Extensive experiments on widely used benchmarks, including COCO Image Captioning, POPE, and MME, demonstrate that AGE consistently mitigates hallucinations across diverse LVLMs such as LLaVA, MiniGPT-4, and mPLUG-Owl2, without additional training or loss of fluency.
Our results highlight that imitating truth-grounded attention dynamics is a simple yet powerful principle to improve the reliability of LVLMs.
Bound by semanticity: universal laws governing the generalization-identification tradeoff
Authors: Marco Nurisso; Jesseba Fernando; Raj Deshpande; Alan Perotti; Raja Marjieh; Steven M Frankland; Richard Lewis; Taylor Whittington Webb; Declan Iain Campbell; Francesco Vaccarino; Jonathan D. Cohen; Giovanni Petri
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Intelligent systems must form internal representations that support both broad generalization and precise identification. 
Here, we show that these two goals are fundamentally in tension with one another. 
We derive closed-form expressions proving that any model whose representations have a finite semantic resolution, impairing long-range similarity computations, must lie on a universal Pareto front linking its probability of correct generalization $p_S$ and identification $p_I$.
We extend this analysis to general input spaces and to parallel processing scenarios, predicting a sharp $1/n$ collapse in the capacity of processing multiple inputs at the same time.
A minimal ReLU network reproduces these laws: a resolution boundary emerges during learning, and empirical $(p_S,p_I)$ trajectories closely match the theory for linearly decaying similarity.
Finally, we show that the same limits appear in far more complex systems, including a convolutional neural network and state-of-the-art vision–language models, indicating that learned finite-resolution similarity are broad and foundational informational constraints rather than toy-model artifacts.
Together, these results provide a precise theory of the generalization–identification tradeoff and clarify how semantic resolution shapes the representational capacity of deep networks and brains alike.
Beyond Magnitude: Leveraging Direction of RLVR Updates for LLM Reasoning
Authors: Kexin Huang; Haoming Meng; Junkang Wu; Jinda Lu; Chiyu Ma; Ziqian Chen; Xue Wang; Bolin Ding; Jiancan Wu; Xiang Wang; Xiangnan He; Guoyin Wang; Jingren Zhou
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Reinforcement learning with verifiable rewards (RLVR) has substantially improved the reasoning capabilities of large language models. 
While existing analyses identify that RLVR-induced changes are sparse, they primarily focus on the **magnitude** of these updates, largely overlooking their **direction**. 
In this work, we argue that the direction of updates is a more critical lens for understanding RLVR's effects, which can be captured by the signed, token-level log probability difference $\Delta\log p$ between the base and final RLVR models.
Through statistical analysis and token-replacement interventions, we demonstrate that $\Delta\log p$ more effectively identifies sparse, yet reasoning-critical updates than magnitude-based metrics (e.g., divergence or entropy).
Building on this insight, we propose two practical applications:
(1) a *test-time extrapolation* method that amplifies the policy along the learned $\Delta\log p$ direction to improve reasoning accuracy without further training;
(2) a *training-time reweighting* method that focuses learning on low-probability (corresponding to higher $\Delta\log p$) tokens, which improves reasoning performance across models and benchmarks.
Our work establishes the direction of change as a key principle for analyzing and improving RLVR.
R2PS: Worst-Case Robust Real-Time Pursuit Strategies under Partial Observability
Authors: Runyu Lu; Ruochuan Shi; Yuanheng Zhu; Dongbin Zhao
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Computing worst-case robust strategies in pursuit-evasion games (PEGs) is time-consuming, especially when real-world factors like partial observability are considered. While important for general security purposes, real-time applicable pursuit strategies for graph-based PEGs are currently missing when the pursuers only have imperfect information about the evader's position. Although state-of-the-art reinforcement learning (RL) methods like Equilibrium Policy Generalization (EPG) and Grasper provide guidelines for learning graph neural network (GNN) policies robust to different game dynamics, they are restricted to the scenario of perfect information and do not take into account the possible case where the evader can predict the pursuers' actions. This paper introduces the first approach to worst-case robust real-time pursuit strategies (R2PS) under partial observability. We first prove that a traditional dynamic programming (DP) algorithm for solving Markov PEGs maintains optimality under the asynchronous moves by the evader. Then, we propose a belief preservation mechanism about the evader's possible positions, extending the DP pursuit strategies to a partially observable setting. Finally, we embed the belief preservation into the state-of-the-art EPG framework to finish our R2PS learning scheme, which leads to a real-time pursuer policy through cross-graph reinforcement learning against the asynchronous-move DP evasion strategies. After reinforcement learning, our policy achieves robust zero-shot generalization to unseen real-world graph structures and consistently outperforms the policy directly trained on the test graphs by the existing game RL approach.
MetaMuse: Algorithm Generation via Creative Ideation
Authors: Ruiying Ma; Chieh-Jan Mike Liang; Yanjie Gao; Francis Y. Yan
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Designing system algorithms remains challenging, where the discontinuous nature of the solution space often forces system engineers to rely on generic heuristics at the expense of performance. We study whether LLMs can practically drive algorithm generation, and find that they are biased towards well-known generic designs, rather than making the creative leaps needed to navigate the discontinuous solution space. To address this limitation, we introduce MetaMuse, a framework for creative ideation built on three self-reflection principles: (1) quantifying solution diversity and usefulness in measurable performance space, rather than abstract idea space, (2) steering ideation through external stimuli, rather than internal randomness, and (3) constructing executable solutions using waypoint reasoning, rather than free-form chain-of-thought. Considering two critical online problems at a global cloud provider, extensive evaluations show that MetaMuse can generate high-performing solutions: it reduces cache misses by up to 35.76%
in cache replacement and reduces bin usage by up to 30.93% in online bin packing.
VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimization
Authors: Jiajing Lin; Shu Jiang; Qingyuan Zeng; Zhenzhong Wang; Min Jiang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The intrinsic dynamics of an object governs its physical behavior in the real world, playing a critical role in enabling physically plausible interactive simulation with 3D assets. Existing methods have attempted to infer the intrinsic dynamics of objects from visual observations, but generally face two major challenges: one line of work relies on manually defined constitutive priors, making it difficult to align with actual intrinsic dynamics; the other models intrinsic dynamics using neural networks, resulting in limited interpretability and poor generalization. To address these challenges, we propose VisionLaw, a bilevel optimization framework that infers interpretable expressions of intrinsic dynamics from visual observations. At the upper level, we introduce an LLMs-driven decoupled constitutive evolution strategy, where LLMs are prompted to act as physics experts to generate and revise constitutive laws, with a built-in decoupling mechanism that substantially reduces the search complexity of LLMs. At the lower level, we introduce a vision-guided constitutive evaluation mechanism, which utilizes visual simulation to evaluate the consistency between the generated constitutive law and the underlying intrinsic dynamics, thereby guiding the upper-level evolution. Experiments on both synthetic and real-world datasets demonstrate that VisionLaw can effectively infer interpretable intrinsic dynamics from visual observations. It significantly outperforms existing state-of-the-art methods and exhibits strong generalization for interactive simulation in novel scenarios.
Overshoot and Shrinkage in Classifier-Free Guidance: From Theory to Practice
Authors: Krunoslav Lehman Pavasovic; Jakob Verbeek; Giulio Biroli; Marc Mezard
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Classifier-Free Guidance (CFG) is widely used in diffusion and flow-based generative models for high-quality conditional generation, yet its theoretical properties remain incompletely understood. By connecting CFG to the high-dimensional framework of diffusion regimes, we show that in sufficiently high dimensions it reproduces the correct target distribution—a “blessing-of-dimensionality” result. Leveraging this theoretical framework, we analyze how the well-known artifacts of mean overshoot and variance shrinkage emerge in lower dimensions, characterizing how they become more pronounced as dimensionality decreases. Building on these insights, we propose a simple nonlinear extension of CFG, proving that it mitigates both effects while preserving CFG’s practical benefits. Finally, we validate our approach through numerical simulations on Gaussian mixtures and real-world experiments on diffusion and flow-matching state-of-the-art class-conditional and text-to-image models, demonstrating continuous improvements in sample quality, diversity, and consistency.
Flow of Spans: Generalizing Language Models to Dynamic Span-Vocabulary via GFlowNets
Authors: Bo Xue; Yunchong Song; Fanghao Shao; Xuekai Zhu; Lin Chen; Luoyi Fu; Xinbing Wang; Zhouhan Lin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Standard autoregressive language models generate text token-by-token from a fixed vocabulary, inducing a *tree-structured state space* when viewing token sampling as an action, which limits flexibility and expressiveness. Recent work introduces dynamic vocabulary by sampling retrieved text spans but overlooks that the same sentence can be composed of spans of varying lengths, lacking explicit modeling of the *directed acyclic graph (DAG) state space*. This leads to restricted exploration of compositional paths and is biased toward the chosen path. Generative Flow Networks (GFlowNets) are powerful for efficient exploring and generalizing over state spaces, particularly those with a DAG structure. However, prior GFlowNets-based language models operate at the token level and remain confined to tree-structured spaces, limiting their potential. In this work, we propose **F**low **o**f **S**pan**S** (**FOSS**), a principled GFlowNets framework for span generation. FoSS constructs a dynamic span vocabulary by segmenting the retrieved text flexibly, ensuring a DAG-structured state space, which allows GFlowNets to explore diverse compositional paths and improve generalization. With specialized reward models, FoSS generates diverse, high-quality text. Empirically, FoSS improves MAUVE scores by up to 12.5\% over Transformer on text generation and achieves 3.5\% gains on knowledge-intensive tasks, consistently outperforming state-of-the-art methods. Scaling experiments further demonstrate FoSS benefits from larger models, more data, and richer retrieval corpora, retaining its advantage over strong baselines.
TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
Authors: Pengfei He; Zhenwei Dai; Bing He; Hui Liu; Xianfeng Tang; Hanqing Lu; Juanhui Li; Jiayuan Ding; Subhabrata Mukherjee; Suhang Wang; Yue Xing; Jiliang Tang; Benoit Dumoulin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large language model (LLM)-based agents increasingly rely on tool use to complete real-world tasks. While existing works evaluate the LLMs' tool use capability, they largely focus on the final answers yet overlook the detailed tool usage trajectory, i.e., whether tools are selected, parameterized, and ordered correctly. We introduce TRAJECT-Bench, a trajectory-aware benchmark to comprehensively evaluate LLMs' tool use capability through diverse tasks with fine-grained evaluation metrics. TRAJECT-Bench pairs high-fidelity, executable tools across practical domains with tasks grounded in production-style APIs, and synthesizes trajectories that vary in breadth (parallel calls) and depth (interdependent chains). Besides final accuracy, TRAJECT-Bench also reports trajectory-level diagnostics, including tool selection and argument correctness, and dependency/order satisfaction. Analyses reveal failure modes such as similar tool confusion and parameter-blind selection, and scaling behavior with tool diversity and trajectory length where the bottleneck of transiting from short to mid-length trajectories is revealed, offering actionable guidance for LLMs' tool use.
Can Large Language Models Match the Conclusions of Systematic Reviews?
Authors: Christopher Polzak; Alejandro Lozano; Min Woo Sun; James Burgess; Yuhui Zhang; Kevin Wu; Chia-Chun Chiang; Jeffrey J Nirschl; Serena Yeung-Levy
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Systematic reviews (SR), in which experts summarize and analyze evidence across individual studies to provide insights on a specialized topic, are a cornerstone for evidence-based clinical decision-making, research, and policy. Given the exponential growth of scientific articles, there is growing interest in using large language models (LLMs) to automate SR generation. However, the ability of LLMs to critically assess evidence and reason across multiple documents to provide recommendations at the same proficiency as domain experts remains poorly characterized. We therefore ask: **Can LLMs match the conclusions of systematic reviews written by clinical experts when given access to the same studies?** To explore this question, we present MedEvidence, a benchmark pairing findings from 100 medical SRs with the studies they are based on. We benchmark 25 LLMs on MedEvidence, including reasoning, non-reasoning, medical specialists, and models across varying sizes (from 7B-700B). Through our systematic evaluation, we find that reasoning does not necessarily improve performance, larger models do not consistently yield greater gains, and knowledge-based fine-tuning degrades accuracy on MedEvidence. Instead, most models exhibit similar behavior: performance tends to degrade as token length increases, their responses show overconfidence, and, contrary to human experts, all models show a lack of scientific skepticism toward low-quality findings. These results suggest that more work is still required before LLMs can reliably match the observations from expert-conducted SRs, even though these systems are already deployed and being used by clinicians.
Go Beyond Earth: Understanding Human Actions and Scenes in Microgravity Environments
Authors: Di Wen; Lei Qi; Kunyu Peng; Kailun Yang; Fei Teng; Ao Luo; Jia Fu; Yufan Chen; Ruiping Liu; Yitian Shi; M. Saquib Sarfraz; Rainer Stiefelhagen
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Despite substantial progress in video understanding, most existing datasets are limited to Earth’s gravitational conditions. However, microgravity alters human motion, interactions, and visual semantics, revealing a critical gap for real-world vision systems. This presents a challenge for domain-robust video understanding in safety-critical space applications.
To address this, we introduce MicroG-4M, the first benchmark for spatio-temporal and semantic understanding of human activities in microgravity. Constructed from real-world space missions and cinematic simulations, the dataset includes $4{,}759$ clips with $13{,}261$ action annotations covering $50$ actions, $1{,}238$ context-rich captions, and over $7{,}000$ question–answer pairs on astronaut activities and scene understanding. MicroG-4M aims to support three core tasks: fine-grained multi-label action recognition, temporal video captioning, and visual question answering, thereby enabling a comprehensive evaluation of both spatial localization and semantic reasoning in microgravity contexts. We establish baselines using state-of-the-art models. All data, annotations, and code are available at https://github.com/lei-qi-233/MicroG-4M.
FlexRibbon: Joint Sequence and Structure Pretraining for Protein Modeling
Authors: Jianwei Zhu; Yu Shi; Ran Bi; Peiran Jin; Chang Liu; Zhe Zhang; Haitao Huang; Zekun Guo; Pipi Hu; Fusong Ju; Lin Huang; Xinwei Tai; Chenao Li; Kaiyuan Gao; Xinran Wei; Huanhuan Xia; Jia Zhang; Yaosen Min; Zun Wang; Yusong Wang; Liang He; Haiguang Liu; Tao Qin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Protein foundation models have advanced rapidly, with most approaches falling into two dominant paradigms. Sequence-based language models (e.g., ESM-2) capture sequence semantics at scale, and a number of recent works incorporate structural signals into sequence encoders. MSA-based predictors (e.g., AlphaFold 2/3) achieve accurate folding by exploiting evolutionary couplings, but their reliance on homologous sequences makes them less reliable in highly mutated or alignment-sparse regimes. We present FlexRibbon, a pretrained protein model that jointly learns from amino acid sequences and three-dimensional structures. Our pretraining strategy combines masked language modeling with diffusion-based denoising, enabling bidirectional sequence-structure learning without requiring MSAs. Trained on both experimentally resolved structures and AlphaFold 2 predictions, FlexRibbon captures global folds as well as flexible conformations critical for biological function. Evaluated across diverse tasks spanning interface design, intermolecular interaction prediction, and protein function prediction, FlexRibbon establishes new state-of-the-art performance on 12 different tasks, with particularly strong gains in mutation-rich settings where MSA-based methods often struggle.
Language Models are Injective and Hence Invertible
Authors: Giorgos Nikolaou; Tommaso Mencattini; Donato Crisostomi; Andrea Santilli; Yannis Panagakis; Emanuele Rodolà
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Transformer components such as non-linear activations and normalization are inherently non-injective, suggesting that different inputs could map to the same output and prevent exact recovery of the input from a model’s representations. In this paper, we challenge this view. First, we prove mathematically that transformer language models mapping discrete input sequences to their corresponding sequence of continuous representations are injective and therefore lossless, a property established at initialization and preserved during training. Second, we confirm this result empirically through billions of collision tests on six state-of-the-art language models, and observe no collisions. Third, we operationalize injectivity: we introduce SipIt, the first algorithm that provably and efficiently reconstructs the exact input text from hidden activations, establishing linear-time guarantees and demonstrating exact invertibility in practice. Overall, our work establishes injectivity as a fundamental and exploitable property of language models, with direct implications for transparency, interpretability, and safe deployment.