GenAI4ELab/papercli-papers huggingface dataset:
Decoding Dynamic Visual Experience from Calcium Imaging via Cell-Pattern-Aware Pretraining
Authors: Sangyoon Bae; Mehdi Azabou; Blake Aaron Richards; Jiook Cha
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Neural recordings exhibit a distinctive form of heterogeneity rooted in differences in cell types, intrinsic circuit dynamics, and stochastic stimulus–response variability that goes beyond ordinary dataset variability, mixing statistically regular neurons with highly stochastic, stimulus-contingent ones within the same dataset. This heterogeneity poses a challenge for self-supervised learning (SSL)—learnable statistical regularity—thereby destabilizing representation learning and limiting reliable scaling.
We introduce POYO-CAP (Cell-pattern Aware Pretraining), a biologically grounded hybrid pretraining strategy that first trains with masked reconstruction plus lightweight auxiliary supervision on statistically regular neurons—identified via skewness and kurtosis—and then fine-tunes on more stochastic populations.
On the Allen Brain Observatory dataset, this curriculum yields 12–13\% relative improvements over from-scratch training and enables smooth, monotonic scaling with model size, whereas baselines trained on mixed populations plateau or destabilize. By making statistical predictability an explicit data-selection criterion, POYO-CAP turns neural heterogeneity into a scalable learning advantage for robust neural decoding.
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
Authors: Ngoc Bui; Shubham Sharma; Simran Lamba; Saumitra Mishra; Rex Ying
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Memory and computation remain core bottlenecks in long-horizon LLM inference due to the quadratic cost of self-attention and the ever-growing key-value (KV) cache. Existing strategies for memory-bounded inference, such as quantization, offloading, or heuristic KV eviction, either incur high orchestration costs or rely on unreliable attention-based proxies of importance. We propose TRIM-KV, a novel approach that learns each token’s intrinsic importance at creation time via a lightweight retention gate. Each gate predicts a scalar retention score that decays over time, reflecting the long-term utility of the token for a specific layer and head. Tokens with low scores are evicted when the memory budget is exceeded, ensuring that the cache always contains the most critical tokens. TRIM-KV is trained efficiently through distillation from a frozen LLM combined with a capacity loss, requiring only gate fine-tuning and adding negligible inference overhead. Across mathematical reasoning (GSM8K, MATH-500, AIME24), procedural generation (LongProc), conversational long-memory benchmarks (LongMemEval), and long-context understanding (LongBenchV2 and SCBench), TRIM-KV consistently outperforms strong eviction and learnable retrieval baselines, especially in low-memory regimes. Remarkably, it even surpasses full-cache models in some settings, showing that selective retention can serve as a form of regularization, suppressing noise from uninformative tokens. Qualitative analyses further reveal that learned retention scores align with human intuition, naturally recovering heuristics such as sink tokens, sliding windows, and gist compression without explicit design. Beyond efficiency, retention scores provide insights into layer- and head-specific roles, suggesting a new path toward LLM interpretability.
Bayesian Influence Functions for Hessian-Free Data Attribution
Authors: Philipp Alexander Kreer; Wilson Wu; Maxwell Adam; Zach Furman; Jesse Hoogland
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Classical influence functions face significant challenges when applied to deep neural networks, primarily due to non-invertible Hessians and high-dimensional parameter spaces. We propose the local Bayesian influence function (BIF), an extension of classical influence functions that replaces Hessian inversion with loss landscape statistics that can be estimated via stochastic-gradient MCMC sampling. This Hessian-free approach captures higher-order interactions among parameters and scales efficiently to neural networks with billions of parameters. We demonstrate state-of-the-art results on predicting retraining experiments.
Time-Gated Multi-Scale Flow Matching for Time-Series Imputation
Authors: Hangtian Wang; Mahito Sugiyama
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We address multivariate time–series imputation by learning the velocity field of a
data-conditioned ordinary differential equation (ODE) via flow matching. Our
method, Time-Gated Multi-Scale Flow Matching (TG-MSFM), conditions the
flow on a structured endpoint comprising observed values, a per-time visibility
mask, and short left/right context, processed by a time-aware Transformer whose
self-attention is masked to aggregate only from observed timestamps. To recon-
cile global trends with local details along the trajectory, we introduce time-gated
multi-scale velocity heads on a fixed 1D pyramid and blend them through a time-
dependent gate; a mild anti-aliasing filter stabilizes the finest branch. At inference,
we use a second-order Heun integrator with a per-step data-consistency projection
that keeps observed coordinates exactly on the straight path from the initial noise
to the endpoint, reducing boundary artifacts and drift. Training adopts gap-only
supervision of the velocity on missing data coordinates, with small optional regu-
larizers for numerical stability. Across standard benchmarks, Time-Gated Multi-
Scale Flow Matching attains competitive or improved MSE/MAE with favorable
speed–quality trade-offs, and ablations isolate the contributions of the time-gated
multi-scale heads, masked attention, and the data-consistent ODE integration
Characterizing Human Semantic Navigation in Concept Production as Trajectories in Embedding Space
Authors: Felipe Diego Toro-Hernández; Jesuino Vieira Filho; Rodrigo M. Cabral-Carvalho
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Semantic representations can be framed as a structured, dynamic knowledge space through which humans navigate to retrieve and manipulate meaning. To investigate how humans traverse this geometry, we introduce a framework that represents concept production as navigation through embedding space. Using different transformer text embedding models, we construct participant-specific semantic trajectories based on cumulative embeddings and extract geometric and dynamical metrics, including distance to next, distance to centroid, entropy, velocity, and acceleration. These measures capture both scalar and directional aspects of semantic navigation, providing a computationally grounded view of semantic representation search as movement in a geometric space. We evaluate the framework on four datasets across different languages, spanning different property generation tasks: Neurodegenerative, Swear verbal fluency,  Property listing task in Italian, and in German. Across these contexts, our approach distinguishes between clinical groups and concept types, offering a mathematical framework that requires minimal human intervention compared to typical labor-intensive linguistic pre-processing methods. Comparison with a non-cumulative approach reveals that cumulative embeddings work best for longer trajectories, whereas shorter ones may provide too little context, favoring the non-cumulative alternative. Critically, different embedding models yielded similar results, highlighting similarities between different learned representations despite different training pipelines. By framing semantic navigation as a structured trajectory through embedding space, bridging cognitive modeling with learned representation, thereby establishing a pipeline for quantifying semantic representation dynamics with applications in clinical research, cross-linguistic analysis, and the assessment of artificial cognition. https://github.com/jesuinovieira/semtraj-iclr2026
Inter-Agent Relative Representations for Multi-Agent Option Discovery
Authors: Raul D. Steleac; Mohan Sridharan; David Abel
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Temporally extended actions improve the ability to explore and plan in single-agent settings. In multi-agent settings, the exponential growth of the joint state space with the number of agents makes coordinated behaviours even more valuable. Yet, this same exponential growth renders the design of multi-agent options particularly challenging. Existing multi-agent option discovery methods often sacrifice coordination by producing loosely coupled or fully independent behaviours. Toward addressing these limitations, we describe a novel approach for multi-agent option discovery. Specifically, we propose a joint-state abstraction that compresses the state space while preserving the information necessary to discover strongly coordinated behaviours. Our approach builds on the inductive bias that synchronisation over agent states provides a natural foundation for coordination in the absence of explicit objectives. We first approximate a fictitious state of maximal alignment with the team, the Fermat state, and use it to define a measure of spreadness, capturing team-level misalignment on each individual state dimension. Building on this representation, we then employ a neural graph Laplacian estimator to derive options that capture state synchronisation patterns between agents. We evaluate the resulting options across multiple scenarios in two simulated multi-agent domains, showing that they yield stronger downstream coordination capabilities compared to alternative option discovery methods.
Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning
Authors: Franki NGUIMATSIA TIOFACK; Théotime Le Hellard; Fabian Schramm; Nicolas Perrin-Gilbert; Justin Carpentier
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Offline reinforcement learning often relies on behavior regularization that enforces policies to remain close to the dataset distribution.  However, such approaches fail to distinguish between high-value and low-value actions in their regularization components. We introduce Guided Flow Policy (GFP), which couples a multi-step flow-matching policy with a distilled one-step actor. The actor directs the flow policy through weighted behavior cloning to focus on cloning high-value actions from the dataset rather than indiscriminately imitating all state-action pairs. In turn, the flow policy constrains the actor to remain aligned with the dataset's best transitions while maximizing the critic. This mutual guidance enables GFP to achieve state-of-the-art performance across 144 state and pixel-based tasks from the OGBench, Minari, and D4RL benchmarks, with substantial gains on suboptimal datasets and challenging tasks.
Evidence for Limited Metacognition in LLMs
Authors: Christopher Ackerman
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The possibility of LLM self-awareness and even sentience is gaining increasing public attention and has major safety and policy implications, but the science of measuring them is still in a nascent state. Here we introduce a novel methodology for quantitatively evaluating metacognitive abilities in LLMs. Taking inspiration from research on metacognition in nonhuman animals, our approach eschews model self-reports and instead tests to what degree models can strategically deploy knowledge of internal states. Using two experimental paradigms, we demonstrate that frontier LLMs introduced since early 2024 show increasingly strong evidence of certain metacognitive abilities, specifically the ability to assess and utilize their own confidence in their ability to answer factual and reasoning questions correctly and the ability to anticipate what answers they would give and utilize that information appropriately. We buttress these behavioral findings with an analysis of the token probabilities returned by the models, which suggests the presence of an upstream internal signal that could provide the basis for metacognition. We further find that these abilities 1) are limited in resolution, 2) emerge in context-dependent manners, and 3) seem to be qualitatively different from those of humans. We also report intriguing differences across models of similar capabilities, suggesting that LLM post-training may have a role in developing metacognitive abilities.
UNDERSTANDING TRANSFORMERS FOR TIME SERIES FORECASTING: A CASE STUDY ON MOIRAI
Authors: Dennis Wu; Yihan He; Yuan Cao; Jianqing Fan; Han Liu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We give a comprehensive theoretical analysis of transformers as time series pre-
diction models, with a focus on MOIRAI (Woo et al., 2024). We study its ap-
proximation and generalization capabilities. First, we demonstrate that there exist
transformers that fit an autoregressive model on input univariate time series via
gradient descent. We then analyze MOIRAI, one of the state-of-the-art multivariate
time series prediction models capable of modeling arbitrary number of covariates.
We prove that MOIRAI is capable of automatically fitting autoregressive models
with an arbitrary number of covariates, offering insights into its design and em-
pirical success. For generalization, we establish learning bounds for pretraining
when the data satisfies Dobrushin’s condition. Experiments support our theoretical
findings, highlighting the efficacy of using transformers for time series forecasting.
Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design
Authors: Xingyu Su; Xiner Li; Masatoshi Uehara; Sunwoo Kim; Yulai Zhao; Gabriele Scalia; Ehsan Hajiramezanali; Tommaso Biancalani; Degui Zhi; Shuiwang Ji
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We address the problem of fine-tuning diffusion models for reward-guided generation in biomolecular design. While diffusion models have proven highly effective in modeling complex, high-dimensional data distributions, real-world applications often demand more than high-fidelity generation, requiring optimization with respect to potentially non-differentiable reward functions such as physics-based simulation or rewards based on scientific knowledge. Although RL methods have been explored to fine-tune diffusion models for such objectives, they often suffer from instability, low sample efficiency, and mode collapse due to their on-policy nature. In this work, we propose an iterative distillation-based fine-tuning framework that enables diffusion models to optimize for arbitrary reward functions. Our method casts the problem as policy distillation: it collects off-policy data during the roll-in phase, simulates reward-based soft-optimal policies during roll-out, and updates the model by minimizing the KL divergence between the simulated soft-optimal policy and the current model policy. Our off-policy formulation, combined with KL divergence minimization, enhances training stability and sample efficiency compared to existing RL-based methods. Empirical results demonstrate the effectiveness and superior reward optimization of our approach across diverse tasks in protein, small molecule, and regulatory DNA design.
Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problems
Authors: Szymon Pawlonka; Mikołaj Małkiński; Jacek Mańdziuk
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Bongard Problems (BPs) provide a challenging testbed for abstract visual reasoning (AVR), requiring models to identify visual concepts from just a few examples and describe them in natural language. Early BP benchmarks featured synthetic black-and-white drawings, which might not fully capture the complexity of real-world scenes. Subsequent BP datasets employed real-world images, albeit the represented concepts are identifiable from high-level image features, reducing the task complexity. Differently, the recently released Bongard-RWR dataset aimed at representing abstract concepts formulated in the original BPs using fine-grained real-world images. Its manual construction, however, limited the dataset size to just $60$ instances, constraining evaluation robustness. In this work, we introduce Bongard-RWR+, a BP dataset composed of $5400$ instances that represent original BP abstract concepts using real-world-like images generated via a vision language model (VLM) pipeline. Building on Bongard-RWR, we employ Pixtral-12B to describe manually curated images and generate new descriptions aligned with the underlying concepts, use Flux.1-dev to synthesize images from these descriptions, and manually verify that the generated images faithfully reflect the intended concepts. We evaluate state-of-the-art VLMs across diverse BP formulations, including binary and multiclass classification, as well as textual answer generation. Our findings reveal that while VLMs can recognize coarse-grained visual concepts, they consistently struggle with discerning fine-grained concepts, highlighting limitations in their reasoning capabilities.
R-WoM: Retrieval-augmented World Model For Computer-use Agents
Authors: Kai Mei; Jiang Guo; Shuaichen Chang; Mingwen Dong; Dongkyu Lee; Xing Niu; Jiarong Jiang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Large Language Models (LLMs) can serve as world models to enhance agent decision-making in digital environments by simulating future states and predicting action outcomes, potentially eliminating costly trial-and-error exploration. However, this capability is fundamentally limited by LLM's tendency to hallucination and their reliance on static training knowledge, which could lead to compounding errors that inhibit long-horizon simulations. 
To systematically investigate whether LLMs are appropriate for world modeling, we probe two core capabilities of world models -- *future state prediction* and *reward estimation* -- through three tasks: next-state identification, full-procedure planning alignment, and milestone transition recognition. Our analysis shows that while LLMs effectively capture immediate next states and identify meaningful state transitions, their performance rapidly degrades in full-procedure planning. This highlights LLMs’ limitations in reliably modeling environment dynamics over long horizons. To address these limitations, we propose the Retrieval-augmented World Model (R-WoM), which grounds LLM simulations by incorporating factual, up-to-date knowledge retrieved from external tutorials. Experiments show that R-WoM achieves relative improvements of up to 23.4\% and 16.3\% on the subsets of OSWorld and Webarena compared to baselines, with particular advantage in longer-horizon simulations.
How to Square Tensor Networks and Circuits Without Squaring Them
Authors: Lorenzo Loconte; Adrián Javaloy; Antonio Vergari
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Squared tensor networks (TNs) and their extension as computational graphs---squared circuits---have been used as expressive distribution estimators, yet supporting closed-form marginalization. However, the squaring operation introduces additional complexity when computing the partition function or marginalizing variables, which hinders their applicability in ML. To solve this issue, canonical forms of TNs are parameterized via unitary matrices to simplify the computation of marginals. However, these canonical forms do not apply to circuits, as they can represent factorizations that do not directly map to a known TN. Inspired by the ideas of orthogonality in canonical forms and determinism in circuits enabling tractable maximization, we show how to parameterize squared circuits to overcome their marginalization overhead. Our parameterizations unlock efficient marginalization even in factorizations different from TNs, but encoded as circuits, whose structure would otherwise make marginalization computationally hard. Finally, our experiments on distribution estimation show how our proposed conditions in squared circuits come with no expressiveness loss, while enabling more efficient learning.
Learning Boltzmann Generators via Constrained Mass Transport
Authors: Christopher von Klitzing; Denis Blessing; Henrik Schopmans; Pascal Friederich; Gerhard Neumann
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Efficient sampling from high-dimensional and multimodal unnormalized probability distributions is a central challenge in many areas of science and machine learning. We focus on Boltzmann generators (BGs) that aim to sample the Boltzmann distribution of physical systems, such as molecules, at a given temperature. Classical variational approaches that minimize the reverse Kullback–Leibler divergence are prone to mode collapse, while annealing-based methods, commonly using geometric schedules, can suffer from mass teleportation and rely heavily on schedule tuning. We introduce *Constrained Mass Transport* (CMT), a variational framework that generates intermediate distributions under constraints on both the KL divergence and the entropy decay between successive steps. These constraints enhance distributional overlap, mitigate mass teleportation, and counteract premature convergence. Across standard BG benchmarks and the here introduced *ELIL tetrapeptide*, the largest system studied to date without access to samples from molecular dynamics, CMT consistently surpasses state-of-the-art variational methods, achieving more than 2.5× higher effective sample size while avoiding mode collapse.
SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modeling
Authors: Elisabetta Fedele; Francis Engelmann; Ian Huang; Or Litany; Marc Pollefeys; Leonidas Guibas
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Generative methods for 3D assets have recently achieved remarkable progress, yet providing intuitive and precise control over the object geometry remains a key challenge.Existing approaches predominantly rely on text or image prompts, which often fall short in geometric specificity: language can be ambiguous, and images are difficult to manipulate. In this work, we introduce SpaceControl, a training-free test-time method for explicit spatial control of 3D asset generation. Our approach accepts a wide range of geometric inputs, from coarse primitives to detailed meshes, and integrates seamlessly with modern generative models without requiring any additional training. A control parameter lets users trade off between geometric fidelity and output realism. Extensive quantitative evaluation and user studies demonstrate that SpaceControl outperforms both training-based and optimization-based baselines in geometric faithfulness while preserving high visual quality. Finally, we present an interactive interface for real-time superquadric editing and direct 3D asset generation, enabling seamless use in creative workflows. Project page: https://spacecontrol3d.github.io/