GenAI4ELab/papercli-papers huggingface dataset:
GmNet: Revisiting Gating Mechanisms From A Frequency View
Authors: Yifan Wang; Xu Ma; Yitian Zhang; Yizhou Wang; Zhongruo Wang; Sung-Cheol Kim; Vahid Mirjalili; Vidya Renganathan; Yun Fu
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Lightweight neural networks, essential for on-device applications, often suffer from a low-frequency bias due to their constrained capacity and depth. 
This limits their ability to capture the fine-grained, high-frequency details (e.g., textures, edges) that are crucial for complex computer vision tasks. To address this fundamental limitation, we perform the first systematic analysis of gating mechanisms from a frequency perspective. 
Inspired by the convolution theorem, we show how the interplay between element-wise multiplication and non-linear activation functions within Gated Linear Units (GLUs) provides a powerful mechanism to selectively amplify high-frequency signals, thereby enriching the model's feature representations.
Based on these findings, we introduce the Gating Mechanism Network (GmNet), a simple yet highly effective architecture that incorporates our frequency-aware gating principles into a standard lightweight backbone. The efficacy of our approach is remarkable: without relying on complex training strategies or architectural search, GmNet achieves a new state-of-the-art for efficient models.
SMAGA: Secondary Motion-Aware 3D Clothed Gaussian Avatars from Monocular Videos
Authors: Seungeun Lee; SeungJun Moon; Hah Min Lew; Ji-Su Kang; Gyeong-Moon Park
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Recent advances in neural rendering, particularly 3D Gaussian Splatting (3DGS), have enabled animatable 3D human avatars from single videos with efficient rendering and high fidelity. However, current methods struggle with dynamic appearances, especially in loose garments (e.g., skirts), causing unrealistic cloth motion and needle artifacts. This paper introduces a novel approach to dynamic appearance modeling for 3DGS-based avatars, focusing on loose clothing. We identify two key challenges: (1) limited Gaussian deformation under pre-defined template articulation, and (2) a mismatch between body-template assumptions and the geometry of loose apparel. To address these issues, we propose a motion-aware autoregressive structural deformation framework for Gaussians. We structure Gaussians into an approximate graph and recursively predict structure-preserving updates, yielding realistic, template-free cloth dynamics. Our framework enables robust dynamic appearance modeling under the single-view constraint, producing accurate foreground silhouettes and precise alignment of Gaussian points with clothed shapes. To demonstrate the effectiveness of our method, we introduce an evaluation dataset featuring subjects performing dynamic movements in loose clothing, and extensive experiments validate that our approach significantly outperforms existing 3DGS-based methods in modeling dynamic appearances from monocular videos.
SONATA: Synergistic Coreset Informed Adaptive Temporal Tensor Factorization
Authors: Maolin Wang; Zhiqi Li; Binhao Wang; Xuhui Chen; Tianshuo Wei; Wanyu Wang; Shikai Fang; Ruocheng Guo; Zenglin Xu; Xiangyu Zhao
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Analyzing dynamic tensor streams is fundamentally challenged by complex, evolving temporal dynamics and the need to identify informative data from high-velocity streams. Existing methods often lack the expressiveness to model multi-scale temporal dependencies, limiting their ability to capture evolving patterns. We propose SONATA, a novel framework that unifies expressive dynamic embedding modeling with adaptive coreset selection. SONATA leverages principled machine learning techniques for efficient evaluation of each observation for uncertainty, novelty, influence, and information gain, and dynamically prioritizes learning from the most valuable data using Bellman-inspired optimization. Entity dynamics are modeled with Linear Dynamical Systems and expressive temporal kernels for fine-grained temporal representation. Experiments on synthetic and real-world datasets show that SONATA consistently outperforms state-of-the-art methods in modeling complex temporal patterns and improving predictive accuracy for dynamic tensor streams.
Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMs
Authors: Ziqian Zhong; Aditi Raghunathan
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The releases of powerful open-weight large language models (LLMs) are often not accompanied by access to their full training data. Existing interpretability methods, particularly those based on activations, often require or assume distributionally similar data. This is a significant limitation when detecting and defending against novel potential threats like backdoors, which are by definition out-of-distribution.
In this work, we introduce a new method for understanding, monitoring and controlling fine-tuned LLMs that interprets weights, rather than activations, thereby sidestepping the need for data that is distributionally similar to the unknown training data. We demonstrate that the top singular vectors of the weight difference between a fine-tuned model and its base model correspond to newly acquired behaviors. By monitoring the cosine similarity of activations along these directions, we can detect salient behaviors introduced during fine-tuning with high precision.
For backdoored models that bypass safety mechanisms when a secret trigger is present, our method stops up to 100% of attacks with a false positive rate below 1%. For models that have undergone unlearning, we detect inference on erased topics with accuracy up to 95.42% and can even steer the model to recover "unlearned" information. Besides monitoring, our method also shows potential for pre-deployment model auditing: by analyzing commercial instruction-tuned models (OLMo, Llama, Qwen), we are able to uncover model-specific fine-tuning focus including mathematical problem solving, emoji usage, and Midjourney prompt generation.
MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents
Authors: Xijia Tao; Teng Yihua; Xinxing Su; Xinyu Fu; Jihao Wu; Chaofan Tao; Ziru Liu; Haoli Bai; Rui Liu; Lingpeng Kong
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Existing multimodal browsing benchmarks often fail to require genuine multimodal reasoning, as many tasks can be solved with text-only heuristics without vision-in-the-loop verification. We introduce MMSearch-Plus, a 311-task benchmark that enforces multimodal understanding by requiring extraction and propagation of fine-grained visual cues through iterative image–text retrieval and cross-validation under retrieval noise.
Our curation procedure seeds questions whose answers require extrapolating from spatial cues and temporal traces to out-of-image facts such as events, dates, and venues.
Beyond the dataset, we provide a model-agnostic agent framework with standard browsing tools and a set-of-mark (SoM) module, which lets the agent place marks, crop subregions, and launch targeted image/text searches. SoM enables provenance-aware zoom-and-retrieve and improves robustness in multi-step reasoning.
We evaluated closed- and open-source MLLMs in this framework. The strongest system achieves an end-to-end accuracy of 36.0%, and integrating SoM produces consistent gains in multiple settings, with improvements up to +3.9 points.
From failure analysis, we observe recurring errors in locating relevant webpages and distinguishing between visually similar events. These results underscore the challenges of real-world multimodal search and establish MMSearch-Plus as a rigorous benchmark for advancing agentic MLLMs.
From Utterance to Vividity: Training Expressive Subtitle Translation LLM via Adaptive Local Preference Optimization
Authors: Chaoqun Cui; Shijing Wang; Liangbin Huang; Qingqing Gu; Zhaolong Huang; Xiao Zeng; Wenji Mao
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The rapid development of Large Language Models (LLMs) has significantly enhanced the general capabilities of machine translation. However, as application scenarios become more complex, the limitations of LLMs in vertical domain translations are gradually becoming apparent. In this study, we focus on how to construct translation LLMs that meet the needs of domain customization. We take visual media subtitle translation as our topic and explore how to train expressive and vivid translation LLMs. We investigated the situations of subtitle translation and other domains of literal and liberal translation, verifying the reliability of LLM as reward model and evaluator for translation. Additionally, to train an expressive translation LLM, we constructed and released a multidirectional subtitle parallel corpus dataset and proposed the Adaptive Local Preference Optimization (ALPO) method to address fine-grained preference alignment. Experimental results demonstrate that ALPO achieves outstanding performance in multidimensional evaluation of translation quality.
Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps
Authors: Shan Wang; Peixia Li; Chenchen Xu; Ziang Cheng; Jiayu Yang; Hongdong Li; Pulak Purkait
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: We propose Light–Geometry Interaction (LGI) maps, a novel representation that encodes light-aware occlusion from monocular depth. Unlike ray tracing, which requires full 3D reconstruction, LGI captures essential light–shadow interactions reliably and accurately, computed from off-the-shelf 2.5D depth map predictions. LGI explicitly ties illumination direction to geometry, providing a physics-inspired prior that constrains generative models. Without such prior,  these models often produce floating shadows, inconsistent illumination, and implausible shadow geometry. Building on this representation, we propose a unified pipeline for joint shadow generation and relighting-unlike prior methods that treat them as disjoint tasks-capturing the intrinsic coupling of illumination and shadowing essential for modeling indirect effects. By embedding LGI into a bridge-matching generative backbone, we reduce ambiguity and enforce physically consistent light–shadow reasoning. To enable effective training, we curated the first large-scale benchmark dataset for joint shadow and relighting, covering reflections, transparency, and complex interreflections. Experiments show significant gains in realism and consistency across synthetic and real images. LGI thus bridges geometry-inspired rendering with generative modeling, enabling efficient, physically consistent shadow generation and relighting.
Towards a Theoretical Understanding of In-context Learning: Stability and Non-I.I.D Generalisation
Authors: Yingjie Wang; Yutian Zhou; Shi Fu; Yuzhu Chen; Yongcheng Jing; Leszek Rutkowski; Dacheng Tao
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: In-context learning (ICL) has demonstrated significant performance improvements in transformer-based large models. This study identifies two key factors influencing ICL generalisation under complex non-i.i.d. scenario: algorithmic stability and distributional discrepancy. First, we establish a stability bound for transformer-based models trained with mini-batch gradient descent, revealing how specific optimization configurations interact with the smoothness of the loss landscape to ensure the stability of non-linear Transformers. Next, we introduce a distribution-level discrepancy measure that highlights the importance of aligning the ICL prompt distribution with the training data distribution to achieve effective generalisation. Building on these  insights, we derive a generalisation error bound for ICL with asymptotic convergence guarantees, which further reveals that token-wise prediction errors accumulate over time and even lead to generalisation collapse if the prediction length is not properly constrained. Finally,  empirical evaluations are provided to validate our theoretical findings.
VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Pottery
Authors: Nonghai Zhang; Zeyu Zhang; Jiazi Wang; Yang Zhao; Hao Tang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Vision-Language Models (VLMs) have achieved significant progress in multimodal understanding tasks, demonstrating strong capabilities particularly in general tasks such as image captioning and visual reasoning. However, when dealing with specialized cultural heritage domains like 3D vase artifacts, existing models face severe data scarcity issues and insufficient domain knowledge limitations. Due to the lack of targeted training data, current VLMs struggle to effectively handle such culturally significant specialized tasks. To address these challenges, we propose the VaseVQA-3D dataset, which serves as the first 3D visual question answering dataset for ancient Greek pottery analysis, collecting 664 ancient Greek vase 3D models with corresponding question-answer data and establishing a complete data construction pipeline. We further develop the VaseVLM model, enhancing model performance in vase artifact analysis through domain-adaptive training. Experimental results validate the effectiveness of our approach, where our VaseVLM-7B-RL achieves 12.8\% improvement in R@1 accuracy and 6.6\% improvement in lexical similarity compared to the strongest baselines on the VaseVQA-3D dataset, significantly improving the recognition and understanding of 3D vase artifacts, providing new technical pathways for digital heritage preservation research.
Unveiling Downstream Performance Scaling of LLMs: A Clustering-Based Perspective
Authors: Chengyin Xu; Kaiyuan Chen; Xiao Li; Ke Shen; Chenggang Li
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The escalating scale and cost of Large Language Models (LLMs) training necessitate accurate pre-training prediction of downstream task performance for comprehensive understanding of scaling properties. This is challenged by: 1) the emergence phenomenon, where unpredictable capabilities appearing suddenly at critical model scales; and 2) uneven task difficulty and inconsistent performance scaling patterns, leading to high metric variability. Current prediction methods lack accuracy and reliability. We propose a Clustering-On-Difficulty (COD) framework for downstream performance prediction. The COD framework clusters tasks by their difficulty scaling features, thereby constructing a more stable and predictable task subset that exhibits well-behaved scaling characteristics with the increase of compute budget. We adopt a performance scaling law to predict cluster-wise performance with theoretical support. Predictable subset performance acts as an intermediate predictor for the full evaluation set. We further derive a mapping function to accurately extrapolate the performance of the subset to the full set. Applied to an LLM with 70B parameters, COD achieved a 1.55\% average prediction error across eight key LLM benchmarks, thus providing actionable insights for scaling properties and training monitoring during LLM pre-training.
Interleave-VLA: Enhancing Robot Manipulation with Image-Text Interleaved Instructions
Authors: Cunxin Fan; Xiaosong Jia; Yihang Sun; Yixiao Wang; Jianglan Wei; Ziyang Gong; Xiangyu Zhao; Masayoshi Tomizuka; Xue Yang; Junchi Yan; Mingyu Ding
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: The rise of foundation models paves the way for generalist robot policies in the physical world. Existing methods relying on text-only instructions often struggle to generalize to unseen scenarios. We argue that interleaved image-text inputs offer richer and less biased context and enable robots to better handle unseen tasks with more versatile human-robot interaction.
Building on this insight, we introduce Interleave-VLA, a robot learning paradigm extending interleaved image-text instructions from digital world to directly generating continuous action sequences in the physical world.
Interleave-VLA offers a natural, flexible, and model-agnostic paradigm that extends state-of-the-art vision-language-action (VLA) models with minimal modifications while achieving strong zero-shot generalization.
Interleave-VLA also includes an automatic pipeline that converts text instructions from Open X-Embodiment into interleaved image-text instructions, resulting in a large-scale real-world interleaved embodied dataset with 210k episodes.
Comprehensive evaluation in simulation and the real world shows that Interleave-VLA offers two major benefits: (1) improves out-of-domain generalization to unseen objects by 2× compared to text input baselines, (2) supports flexible task interfaces and diverse instructions in a zero-shot manner, such as hand-drawn sketches.
We attribute Interleave-VLA's strong zero-shot capability to the use of instruction images, which effectively mitigate hallucinations, and the inclusion of heterogeneous multimodal datasets, enriched with Internet-sourced images, offering potential for scalability. [Our project site](https://interleave-vla.github.io/Interleave-VLA-Anonymous/) has more information.
mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules
Authors: Carl Edwards; Chi Han; Gawon Lee; Thao Nguyen; Sara Szymkuć; Chetan Kumar Prasad; Bowen Jin; Jiawei Han; Ying Diao; Ge Liu; Hao Peng; Bartosz Andrzej Grzybowski; Martin D. Burke; Heng Ji
Venue: ICLR 2026 (ICLR 2026 Oral)

Abstract: Despite their ability to understand chemical knowledge, large language models (LLMs) remain limited in their capacity to propose novel molecules with desired functions (e.g., drug-like properties). In addition, the molecules that LLMs propose can often be challenging to make, and are almost never compatible with automated synthesis approaches. To better enable the discovery of functional small molecules, LLMs need to learn a new molecular language that is more effective in predicting properties and inherently synced with automated synthesis technology. Current molecule LLMs are limited by representing molecules based on atoms. In this paper, we argue that just like tokenizing texts into meaning-bearing (sub-)word tokens instead of characters, molecules should be tokenized at the level of functional building blocks, i.e., parts of molecules that bring unique functions and serve as effective building blocks for real-world automated laboratory synthesis. This motivates us to propose mCLM, a modular Chemical-Language Model that comprises a bilingual language model that understands both natural language descriptions of functions and molecular blocks. mCLM front-loads synthesizability considerations while improving the predicted functions of molecules in a principled manner. Experiments on 430 FDA-approved drugs showed that mCLM is capable of significantly improving chemical functions critical to determining drug potentials. mCLM, with only 3B parameters, also achieves improvements in synthetic accessibility relative to 7 other leading generative AI methods including GPT-5. When tested on 122 out-of-distribution medicines using only building blocks/tokens that are compatible with automated modular synthesis, mCLM outperforms all baselines in property scores and synthetic accessibility. mCLM can also reason on multiple functions and iteratively self-improve to rescue drug candidates that failed late in clinical trials (“fallen angels”).
CAD-Tokenizer: Towards Text-Based CAD Prototyping via Modality-Specific Tokenization
Authors: Ruiyu Wang; Shizhao Sun; Weijian Ma; Jiang Bian
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Computer-Aided Design (CAD) is a foundational component of industrial prototyping. 
where models are defined not by raw coordinates but by construction sequences such as sketches and extrusions. 
This sequential structure enables both efficient prototype initialization and subsequent editing. 
Text-guided CAD prototyping, which unifies Text-to-CAD generation and CAD editing, has the potential to streamline the entire design pipeline. 
However, prior work has not explored this setting, largely because standard large language model (LLM) tokenizers decompose CAD sequences into natural-language word pieces, failing to capture primitive-level CAD semantics and hindering attention modules from modeling geometric structure.
We conjecture that a multimodal tokenization strategy, aligned with CAD’s primitive and structural nature, can provide more effective representations. 
To this end, we propose CAD-Tokenizer, a framework that represents CAD data with modality-specific tokens using a sequence-based VQ-VAE with primitive-level pooling and constrained decoding.
This design produces compact, primitive-aware representations that align with CAD’s structural nature. 
Applied to unified text-guided CAD prototyping, CAD-Tokenizer significantly improves instruction following and generation quality, achieving better quantitative and qualitative performance over both general-purpose LLMs and task-specific baselines.
PonderLM: Pretraining Language Models to Ponder in Continuous Space
Authors: Boyi Zeng; Shixiang Song; Siyuan Huang; Yixuan Wang; He Li; Ziwei He; Xinbing Wang; Zhiyu li; Zhouhan Lin
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Humans ponder before articulating complex sentence elements, enabling deeper cognitive processing through focused effort.
In this work, we introduce this pondering process into language models by repeatedly invoking the forward process within a single token generation step. During pondering, instead of generating an actual token sampled from the prediction distribution, the model ponders by yielding a weighted sum of all token embeddings according to the predicted token distribution. The generated embedding is then fed back as input for another forward pass. We show that the model can learn to ponder in this way through self-supervised learning, without any human annotations.
Experiments across three widely used open-source architectures—GPT-2, Pythia, and LLaMA—and extensive downstream task evaluations demonstrate the effectiveness and generality of our method. On 9 downstream benchmarks, our pondering-enhanced Pythia models significantly outperform the official Pythia models. Notably, our PonderPythia models demonstrate remarkable effectiveness: PonderPythia-2.8B surpasses Pythia-6.9B and rivals Pythia-12B, while our PonderPythia-1B matches TinyLlama-1.1B, a model trained on 10 times more data.
MedLesionVQA: A Multimodal Benchmark Emulating Clinical Visual Diagnosis for Body Surface Health
Authors: Deli Yu; Shengzhi Wang; Kai WU; Xiaozhong Ji; Bo Cui; Jieqiong Cao; Huichao Wang; Boyuan Jiang; Xu Wang; Qian Xu; ChaoGao; Yi Zhao; Dian Chen; Meng Li; Haifeng Wu; Yijun He; Haihua Yang
Venue: ICLR 2026 (ICLR 2026 Poster)

Abstract: Body-surface health conditions, spanning diverse clinical departments, represent some of
the most frequent diagnostic scenarios and a primary target for medical multimodal large
language models (MLLMs). Yet existing medical benchmarks are either built from publicly
available sources with limited expert curation or focus narrowly on disease classification,
failing to reflect the stepwise recognition and reasoning processes physicians follow in
real practice. To address this gap, we introduce MedLesionVQA, the first benchmark
explicitly designed to evaluate MLLMs on the visual diagnostic workflow for body-surface
conditions in large scale. All questions are derived from authentic clinical visual diagnosis
scenarios and verified by medical experts with over 20 years of experience, while the
data are drawn from 10k+ real patient visits, ensuring authenticity, clinical reality and
diversity. MedLesionVQA consists of 12K in-house images (never publicly leaked) and
19K expert-verified question–answer pairs, with fine-grained annotations of 94 lesion types,
110 body regions, and 96 diseases. We evaluate 20+ state-of-the-art MLLMs against
human physicians: the best model reaches 56.2% accuracy, far below primary physicians
(61.4%) and senior specialists (73.2%). These results expose the persistent gap between
MLLMs and clinical expertise, underscoring the need for the multimodal benchmarks to
drive trustworthy medical AI. The dataset can be found in https://github.com/bytedance/MedLesionVQA.