Orionnnn/hub-stats huggingface dataset:
Improving Chemical Understanding of LLMs via SMILES Parsing
Authors: name: Yunhui Jang, name: Jaehyung Kim, name: Sungsoo Ahn
Published: 2025-05-22 07:54:39

Large language models (LLMs) are increasingly recognized as powerful tools
for scientific discovery, particularly in molecular science. A fundamental
requirement for these models is the ability to accurately understand molecular
structures, commonly encoded in the SMILES representation. However, current
LLMs struggle to interpret SMILES, even failing to carry out basic tasks such
as counting molecular rings. To address this limitation, we introduce CLEANMOL,
a novel framework that formulates SMILES parsing into a suite of clean and
deterministic tasks explicitly designed to promote graph-level molecular
comprehension. These tasks span from subgraph matching to global graph
matching, providing structured supervision aligned with molecular structural
properties. We construct a molecular pretraining dataset with adaptive
difficulty scoring and pre-train open-source LLMs on these tasks. Our results
show that CLEANMOL not only enhances structural comprehension but also achieves
the best or competes with the baseline on the Mol-Instructions benchmark.
Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM
  Reasoning
Authors: name: Michael Hassid, name: Gabriel Synnaeve, name: Yossi Adi, name: Roy Schwartz
Published: 2025-05-23 12:29:06

Reasoning large language models (LLMs) heavily rely on scaling test-time
compute to perform complex reasoning tasks by generating extensive "thinking"
chains. While demonstrating impressive results, this approach incurs
significant computational costs and inference time. In this work, we challenge
the assumption that long thinking chains results in better reasoning
capabilities. We first demonstrate that shorter reasoning chains within
individual questions are significantly more likely to yield correct answers -
up to 34.5% more accurate than the longest chain sampled for the same question.
Based on these results, we suggest short-m@k, a novel reasoning LLM inference
method. Our method executes k independent generations in parallel and halts
computation once the first m thinking processes are done. The final answer is
chosen using majority voting among these m chains. Basic short-1@k demonstrates
similar or even superior performance over standard majority voting in
low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while
slightly less efficient than short-1@k, consistently surpasses majority voting
across all compute budgets, while still being substantially faster (up to 33%
wall time reduction). Inspired by our results, we finetune an LLM using short,
long, and randomly selected reasoning chains. We then observe that training on
the shorter ones leads to better performance. Our findings suggest rethinking
current methods of test-time compute in reasoning LLMs, emphasizing that longer
"thinking" does not necessarily translate to improved performance and can,
counter-intuitively, lead to degraded results.
DetailFlow: 1D Coarse-to-Fine Autoregressive Image Generation via
  Next-Detail Prediction
Authors: name: Yiheng Liu, name: Liao Qu, name: Huichao Zhang, name: Xu Wang, name: Yi Jiang, name: Yiming Gao, name: Hu Ye, name: Xian Li, name: Shuai Wang, name: Daniel K. Du, name: Shu Cheng, name: Zehuan Yuan, name: Xinglong Wu
Published: 2025-05-27 17:45:21

This paper presents DetailFlow, a coarse-to-fine 1D autoregressive (AR) image
generation method that models images through a novel next-detail prediction
strategy. By learning a resolution-aware token sequence supervised with
progressively degraded images, DetailFlow enables the generation process to
start from the global structure and incrementally refine details. This
coarse-to-fine 1D token sequence aligns well with the autoregressive inference
mechanism, providing a more natural and efficient way for the AR model to
generate complex visual content. Our compact 1D AR model achieves high-quality
image synthesis with significantly fewer tokens than previous approaches, i.e.
VAR/VQGAN. We further propose a parallel inference mechanism with
self-correction that accelerates generation speed by approximately 8x while
reducing accumulation sampling error inherent in teacher-forcing supervision.
On the ImageNet 256x256 benchmark, our method achieves 2.96 gFID with 128
tokens, outperforming VAR (3.3 FID) and FlexVAR (3.05 FID), which both require
680 tokens in their AR models. Moreover, due to the significantly reduced token
count and parallel inference mechanism, our method runs nearly 2x faster
inference speed compared to VAR and FlexVAR. Extensive experimental results
demonstrate DetailFlow's superior generation quality and efficiency compared to
existing state-of-the-art methods.
Adversarial Attacks against Closed-Source MLLMs via Feature Optimal
  Alignment
Authors: name: Xiaojun Jia, name: Sensen Gao, name: Simeng Qin, name: Tianyu Pang, name: Chao Du, name: Yihao Huang, name: Xinfeng Li, name: Yiming Li, name: Bo Li, name: Yang Liu
Published: 2025-05-27 17:56:57

Multimodal large language models (MLLMs) remain vulnerable to transferable
adversarial examples. While existing methods typically achieve targeted attacks
by aligning global features-such as CLIP's [CLS] token-between adversarial and
target samples, they often overlook the rich local information encoded in patch
tokens. This leads to suboptimal alignment and limited transferability,
particularly for closed-source models. To address this limitation, we propose a
targeted transferable adversarial attack method based on feature optimal
alignment, called FOA-Attack, to improve adversarial transfer capability.
Specifically, at the global level, we introduce a global feature loss based on
cosine similarity to align the coarse-grained features of adversarial samples
with those of target samples. At the local level, given the rich local
representations within Transformers, we leverage clustering techniques to
extract compact local patterns to alleviate redundant local features. We then
formulate local feature alignment between adversarial and target samples as an
optimal transport (OT) problem and propose a local clustering optimal transport
loss to refine fine-grained feature alignment. Additionally, we propose a
dynamic ensemble model weighting strategy to adaptively balance the influence
of multiple models during adversarial example generation, thereby further
improving transferability. Extensive experiments across various models
demonstrate the superiority of the proposed method, outperforming
state-of-the-art methods, especially in transferring to closed-source MLLMs.
The code is released at https://github.com/jiaxiaojunQAQ/FOA-Attack.
MMMR: Benchmarking Massive Multi-Modal Reasoning Tasks
Authors: name: Guiyao Tie, name: Xueyang Zhou, name: Tianhe Gu, name: Ruihang Zhang, name: Chaoran Hu, name: Sizhe Zhang, name: Mengqu Sun, name: Yan Zhang, name: Pan Zhou, name: Lichao Sun
Published: 2025-05-22 09:41:55

Recent advances in Multi-Modal Large Language Models (MLLMs) have enabled
unified processing of language, vision, and structured inputs, opening the door
to complex tasks such as logical deduction, spatial reasoning, and scientific
analysis. Despite their promise, the reasoning capabilities of MLLMs,
particularly those augmented with intermediate thinking traces (MLLMs-T),
remain poorly understood and lack standardized evaluation benchmarks. Existing
work focuses primarily on perception or final answer correctness, offering
limited insight into how models reason or fail across modalities. To address
this gap, we introduce the MMMR, a new benchmark designed to rigorously
evaluate multi-modal reasoning with explicit thinking. The MMMR comprises 1) a
high-difficulty dataset of 1,083 questions spanning six diverse reasoning types
with symbolic depth and multi-hop demands and 2) a modular Reasoning Trace
Evaluation Pipeline (RTEP) for assessing reasoning quality beyond accuracy
through metrics like relevance, consistency, and structured error annotations.
Empirical results show that MLLMs-T overall outperform non-thinking
counterparts, but even top models like Claude-3.7-Sonnet and Gemini-2.5 Pro
suffer from reasoning pathologies such as inconsistency and overthinking. This
benchmark reveals persistent gaps between accuracy and reasoning quality and
provides an actionable evaluation pipeline for future model development.
Overall, the MMMR offers a scalable foundation for evaluating, comparing, and
improving the next generation of multi-modal reasoning systems.
VerIPO: Cultivating Long Reasoning in Video-LLMs via Verifier-Gudied
  Iterative Policy Optimization
Authors: name: Yunxin Li, name: Xinyu Chen, name: Zitao Li, name: Zhenyu Liu, name: Longyue Wang, name: Wenhan Luo, name: Baotian Hu, name: Min Zhang
Published: 2025-05-25 06:41:28

Applying Reinforcement Learning (RL) to Video Large Language Models
(Video-LLMs) shows significant promise for complex video reasoning. However,
popular Reinforcement Fine-Tuning (RFT) methods, such as outcome-based Group
Relative Policy Optimization (GRPO), are limited by data preparation
bottlenecks (e.g., noise or high cost) and exhibit unstable improvements in the
quality of long chain-of-thoughts (CoTs) and downstream performance.To address
these limitations, we propose VerIPO, a Verifier-guided Iterative Policy
Optimization method designed to gradually improve video LLMs' capacity for
generating deep, long-term reasoning chains. The core component is
Rollout-Aware Verifier, positioned between the GRPO and Direct Preference
Optimization (DPO) training phases to form the GRPO-Verifier-DPO training loop.
This verifier leverages small LLMs as a judge to assess the reasoning logic of
rollouts, enabling the construction of high-quality contrastive data, including
reflective and contextually consistent CoTs. These curated preference samples
drive the efficient DPO stage (7x faster than GRPO), leading to marked
improvements in reasoning chain quality, especially in terms of length and
contextual consistency. This training loop benefits from GRPO's expansive
search and DPO's targeted optimization. Experimental results demonstrate: 1)
Significantly faster and more effective optimization compared to standard GRPO
variants, yielding superior performance; 2) Our trained models exceed the
direct inference of large-scale instruction-tuned Video-LLMs, producing long
and contextually consistent CoTs on diverse video reasoning tasks; and 3) Our
model with one iteration outperforms powerful LMMs (e.g., Kimi-VL) and long
reasoning models (e.g., Video-R1), highlighting its effectiveness and
stability.
VisualToolAgent (VisTA): A Reinforcement Learning Framework for Visual
  Tool Selection
Authors: name: Zeyi Huang, name: Yuyang Ji, name: Anirudh Sundara Rajan, name: Zefan Cai, name: Wen Xiao, name: Junjie Hu, name: Yong Jae Lee
Published: 2025-05-26 17:59:17

We introduce VisTA, a new reinforcement learning framework that empowers
visual agents to dynamically explore, select, and combine tools from a diverse
library based on empirical performance. Existing methods for tool-augmented
reasoning either rely on training-free prompting or large-scale fine-tuning;
both lack active tool exploration and typically assume limited tool diversity,
and fine-tuning methods additionally demand extensive human supervision. In
contrast, VisTA leverages end-to-end reinforcement learning to iteratively
refine sophisticated, query-specific tool selection strategies, using task
outcomes as feedback signals. Through Group Relative Policy Optimization
(GRPO), our framework enables an agent to autonomously discover effective
tool-selection pathways without requiring explicit reasoning supervision.
Experiments on the ChartQA, Geometry3K, and BlindTest benchmarks demonstrate
that VisTA achieves substantial performance gains over training-free baselines,
especially on out-of-distribution examples. These results highlight VisTA's
ability to enhance generalization, adaptively utilize diverse tools, and pave
the way for flexible, experience-driven visual reasoning systems.
SeePhys: Does Seeing Help Thinking? -- Benchmarking Vision-Based Physics
  Reasoning
Authors: name: Kun Xiang, name: Heng Li, name: Terry Jingchen Zhang, name: Yinya Huang, name: Zirong Liu, name: Peixin Qu, name: Jixi He, name: Jiaqi Chen, name: Yu-Jie Yuan, name: Jianhua Han, name: Hang Xu, name: Hanhui Li, name: Mrinmaya Sachan, name: Xiaodan Liang
Published: 2025-05-25 11:28:34

We present SeePhys, a large-scale multimodal benchmark for LLM reasoning
grounded in physics questions ranging from middle school to PhD qualifying
exams. The benchmark covers 7 fundamental domains spanning the physics
discipline, incorporating 21 categories of highly heterogeneous diagrams. In
contrast to prior works where visual elements mainly serve auxiliary purposes,
our benchmark features a substantial proportion of vision-essential problems
(75\%) that mandate visual information extraction for correct solutions.
Through extensive evaluation, we observe that even the most advanced visual
reasoning models (e.g., Gemini-2.5-pro and o4-mini) achieve sub-60\% accuracy
on our benchmark. These results reveal fundamental challenges in current large
language models' visual understanding capabilities, particularly in: (i)
establishing rigorous coupling between diagram interpretation and physics
reasoning, and (ii) overcoming their persistent reliance on textual cues as
cognitive shortcuts.
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
Authors: name: Jiakang Yuan, name: Tianshuo Peng, name: Yilei Jiang, name: Yiting Lu, name: Renrui Zhang, name: Kaituo Feng, name: Chaoyou Fu, name: Tao Chen, name: Lei Bai, name: Bo Zhang, name: Xiangyu Yue
Published: 2025-05-27 15:23:23

Logical reasoning is a fundamental aspect of human intelligence and an
essential capability for multimodal large language models (MLLMs). Despite the
significant advancement in multimodal reasoning, existing benchmarks fail to
comprehensively evaluate their reasoning abilities due to the lack of explicit
categorization for logical reasoning types and an unclear understanding of
reasoning. To address these issues, we introduce MME-Reasoning, a comprehensive
benchmark designed to evaluate the reasoning ability of MLLMs, which covers all
three types of reasoning (i.e., inductive, deductive, and abductive) in its
questions. We carefully curate the data to ensure that each question
effectively evaluates reasoning ability rather than perceptual skills or
knowledge breadth, and extend the evaluation protocols to cover the evaluation
of diverse questions. Our evaluation reveals substantial limitations of
state-of-the-art MLLMs when subjected to holistic assessments of logical
reasoning capabilities. Even the most advanced MLLMs show limited performance
in comprehensive logical reasoning, with notable performance imbalances across
reasoning types. In addition, we conducted an in-depth analysis of approaches
such as ``thinking mode'' and Rule-based RL, which are commonly believed to
enhance reasoning abilities. These findings highlight the critical limitations
and performance imbalances of current MLLMs in diverse logical reasoning
scenarios, providing comprehensive and systematic insights into the
understanding and evaluation of reasoning capabilities.
SoloSpeech: Enhancing Intelligibility and Quality in Target Speech
  Extraction through a Cascaded Generative Pipeline
Authors: name: Helin Wang, name: Jiarui Hai, name: Dongchao Yang, name: Chen Chen, name: Kai Li, name: Junyi Peng, name: Thomas Thebaud, name: Laureano Moro Velazquez, name: Jesus Villalba, name: Najim Dehak
Published: 2025-05-25 21:00:48

Target Speech Extraction (TSE) aims to isolate a target speaker's voice from
a mixture of multiple speakers by leveraging speaker-specific cues, typically
provided as auxiliary audio (a.k.a. cue audio). Although recent advancements in
TSE have primarily employed discriminative models that offer high perceptual
quality, these models often introduce unwanted artifacts, reduce naturalness,
and are sensitive to discrepancies between training and testing environments.
On the other hand, generative models for TSE lag in perceptual quality and
intelligibility. To address these challenges, we present SoloSpeech, a novel
cascaded generative pipeline that integrates compression, extraction,
reconstruction, and correction processes. SoloSpeech features a
speaker-embedding-free target extractor that utilizes conditional information
from the cue audio's latent space, aligning it with the mixture audio's latent
space to prevent mismatches. Evaluated on the widely-used Libri2Mix dataset,
SoloSpeech achieves the new state-of-the-art intelligibility and quality in
target speech extraction and speech separation tasks while demonstrating
exceptional generalization on out-of-domain data and real-world scenarios.
GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient
  Fine-Tuning
Authors: name: Yeonjoon Jung, name: Daehyun Ahn, name: Hyungjun Kim, name: Taesu Kim, name: Eunhyeok Park
Published: 2025-05-26 06:48:20

Low-Rank Adaptation (LoRA) is a popular method for parameter-efficient
fine-tuning (PEFT) of generative models, valued for its simplicity and
effectiveness. Despite recent enhancements, LoRA still suffers from a
fundamental limitation: overfitting when the bottleneck is widened. It performs
best at ranks 32-64, yet its accuracy stagnates or declines at higher ranks,
still falling short of full fine-tuning (FFT) performance. We identify the root
cause as LoRA's structural bottleneck, which introduces gradient entanglement
to the unrelated input channels and distorts gradient propagation. To address
this, we introduce a novel structure, Granular Low-Rank Adaptation (GraLoRA)
that partitions weight matrices into sub-blocks, each with its own low-rank
adapter. With negligible computational or storage cost, GraLoRA overcomes
LoRA's limitations, effectively increases the representational capacity, and
more closely approximates FFT behavior. Experiments on code generation and
commonsense reasoning benchmarks show that GraLoRA consistently outperforms
LoRA and other baselines, achieving up to +8.5% absolute gain in Pass@1 on
HumanEval+. These improvements hold across model sizes and rank settings,
making GraLoRA a scalable and robust solution for PEFT. Code, data, and scripts
are available at https://github.com/SqueezeBits/GraLoRA.git
Frame In-N-Out: Unbounded Controllable Image-to-Video Generation
Authors: name: Boyang Wang, name: Xuweiyi Chen, name: Matheus Gadelha, name: Zezhou Cheng
Published: 2025-05-27 17:56:07

Controllability, temporal coherence, and detail synthesis remain the most
critical challenges in video generation. In this paper, we focus on a commonly
used yet underexplored cinematic technique known as Frame In and Frame Out.
Specifically, starting from image-to-video generation, users can control the
objects in the image to naturally leave the scene or provide breaking new
identity references to enter the scene, guided by user-specified motion
trajectory. To support this task, we introduce a new dataset curated
semi-automatically, a comprehensive evaluation protocol targeting this setting,
and an efficient identity-preserving motion-controllable video Diffusion
Transformer architecture. Our evaluation shows that our proposed approach
significantly outperforms existing baselines.
Paper2Poster: Towards Multimodal Poster Automation from Scientific
  Papers
Authors: name: Wei Pang, name: Kevin Qinghong Lin, name: Xiangru Jian, name: Xi He, name: Philip Torr
Published: 2025-05-27 17:58:49

Academic poster generation is a crucial yet challenging task in scientific
communication, requiring the compression of long-context interleaved documents
into a single, visually coherent page. To address this challenge, we introduce
the first benchmark and metric suite for poster generation, which pairs recent
conference papers with author-designed posters and evaluates outputs on
(i)Visual Quality-semantic alignment with human posters, (ii)Textual
Coherence-language fluency, (iii)Holistic Assessment-six fine-grained aesthetic
and informational criteria scored by a VLM-as-judge, and notably
(iv)PaperQuiz-the poster's ability to convey core paper content as measured by
VLMs answering generated quizzes. Building on this benchmark, we propose
PosterAgent, a top-down, visual-in-the-loop multi-agent pipeline: the (a)Parser
distills the paper into a structured asset library; the (b)Planner aligns
text-visual pairs into a binary-tree layout that preserves reading order and
spatial balance; and the (c)Painter-Commenter loop refines each panel by
executing rendering code and using VLM feedback to eliminate overflow and
ensure alignment. In our comprehensive evaluation, we find that GPT-4o
outputs-though visually appealing at first glance-often exhibit noisy text and
poor PaperQuiz scores, and we find that reader engagement is the primary
aesthetic bottleneck, as human-designed posters rely largely on visual
semantics to convey meaning. Our fully open-source variants (e.g. based on the
Qwen-2.5 series) outperform existing 4o-driven multi-agent systems across
nearly all metrics, while using 87% fewer tokens. It transforms a 22-page paper
into a finalized yet editable .pptx poster - all for just $0.005. These
findings chart clear directions for the next generation of fully automated
poster-generation models. The code and datasets are available at
https://github.com/Paper2Poster/Paper2Poster.
Active-O3: Empowering Multimodal Large Language Models with Active
  Perception via GRPO
Authors: name: Muzhi Zhu, name: Hao Zhong, name: Canyu Zhao, name: Zongze Du, name: Zheng Huang, name: Mingyu Liu, name: Hao Chen, name: Cheng Zou, name: Jingdong Chen, name: Ming Yang, name: Chunhua Shen
Published: 2025-05-27 17:29:31

Active vision, also known as active perception, refers to the process of
actively selecting where and how to look in order to gather task-relevant
information. It is a critical component of efficient perception and
decision-making in humans and advanced embodied agents. Recently, the use of
Multimodal Large Language Models (MLLMs) as central planning and
decision-making modules in robotic systems has gained extensive attention.
However, despite the importance of active perception in embodied intelligence,
there is little to no exploration of how MLLMs can be equipped with or learn
active perception capabilities. In this paper, we first provide a systematic
definition of MLLM-based active perception tasks. We point out that the
recently proposed GPT-o3 model's zoom-in search strategy can be regarded as a
special case of active perception; however, it still suffers from low search
efficiency and inaccurate region selection. To address these issues, we propose
ACTIVE-O3, a purely reinforcement learning based training framework built on
top of GRPO, designed to equip MLLMs with active perception capabilities. We
further establish a comprehensive benchmark suite to evaluate ACTIVE-O3 across
both general open-world tasks, such as small-object and dense object grounding,
and domain-specific scenarios, including small object detection in remote
sensing and autonomous driving, as well as fine-grained interactive
segmentation. In addition, ACTIVE-O3 also demonstrates strong zero-shot
reasoning abilities on the V* Benchmark, without relying on any explicit
reasoning data. We hope that our work can provide a simple codebase and
evaluation protocol to facilitate future research on active perception in
MLLMs.