Orionnnn/hub-stats huggingface dataset:
CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic
  Design Generation
Authors: name: Zhao Zhang, name: Yutao Cheng, name: Dexiang Hong, name: Maoke Yang, name: Gonglei Shi, name: Lei Ma, name: Hui Zhang, name: Jie Shao, name: Xinglong Wu
Published: 2025-06-12 16:54:39

Graphic design plays a crucial role in both commercial and personal contexts,
yet creating high-quality, editable, and aesthetically pleasing graphic
compositions remains a time-consuming and skill-intensive task, especially for
beginners. Current AI tools automate parts of the workflow, but struggle to
accurately incorporate user-supplied assets, maintain editability, and achieve
professional visual appeal. Commercial systems, like Canva Magic Design, rely
on vast template libraries, which are impractical for replicate. In this paper,
we introduce CreatiPoster, a framework that generates editable, multi-layer
compositions from optional natural-language instructions or assets. A protocol
model, an RGBA large multimodal model, first produces a JSON specification
detailing every layer (text or asset) with precise layout, hierarchy, content
and style, plus a concise background prompt. A conditional background model
then synthesizes a coherent background conditioned on this rendered foreground
layers. We construct a benchmark with automated metrics for graphic-design
generation and show that CreatiPoster surpasses leading open-source approaches
and proprietary commercial systems. To catalyze further research, we release a
copyright-free corpus of 100,000 multi-layer designs. CreatiPoster supports
diverse applications such as canvas editing, text overlay, responsive resizing,
multilingual adaptation, and animated posters, advancing the democratization of
AI-assisted graphic design. Project homepage:
https://github.com/graphic-design-ai/creatiposter
LaMP-Cap: Personalized Figure Caption Generation With Multimodal Figure
  Profiles
Authors: name: Ho Yin 'Sam' Ng, name: Ting-Yao Hsu, name: Aashish Anantha Ramakrishnan, name: Branislav Kveton, name: Nedim Lipka, name: Franck Dernoncourt, name: Dongwon Lee, name: Tong Yu, name: Sungchul Kim, name: Ryan A. Rossi, name: Ting-Hao 'Kenneth' Huang
Published: 2025-06-06 22:16:16

Figure captions are crucial for helping readers understand and remember a
figure's key message. Many models have been developed to generate these
captions, helping authors compose better quality captions more easily. Yet,
authors almost always need to revise generic AI-generated captions to match
their writing style and the domain's style, highlighting the need for
personalization. Despite language models' personalization (LaMP) advances,
these technologies often focus on text-only settings and rarely address
scenarios where both inputs and profiles are multimodal. This paper introduces
LaMP-Cap, a dataset for personalized figure caption generation with multimodal
figure profiles. For each target figure, LaMP-Cap provides not only the needed
inputs, such as figure images, but also up to three other figures from the same
document--each with its image, caption, and figure-mentioning paragraphs--as a
profile to characterize the context. Experiments with four LLMs show that using
profile information consistently helps generate captions closer to the original
author-written ones. Ablation studies reveal that images in the profile are
more helpful than figure-mentioning paragraphs, highlighting the advantage of
using multimodal profiles over text-only ones.
Optimus-3: Towards Generalist Multimodal Minecraft Agents with Scalable
  Task Experts
Authors: name: Zaijing Li, name: Yuquan Xie, name: Rui Shao, name: Gongwei Chen, name: Weili Guan, name: Dongmei Jiang, name: Liqiang Nie
Published: 2025-06-12 05:29:40

Recently, agents based on multimodal large language models (MLLMs) have
achieved remarkable progress across various domains. However, building a
generalist agent with capabilities such as perception, planning, action,
grounding, and reflection in open-world environments like Minecraft remains
challenges: insufficient domain-specific data, interference among heterogeneous
tasks, and visual diversity in open-world settings. In this paper, we address
these challenges through three key contributions. 1) We propose a
knowledge-enhanced data generation pipeline to provide scalable and
high-quality training data for agent development. 2) To mitigate interference
among heterogeneous tasks, we introduce a Mixture-of-Experts (MoE) architecture
with task-level routing. 3) We develop a Multimodal Reasoning-Augmented
Reinforcement Learning approach to enhance the agent's reasoning ability for
visual diversity in Minecraft. Built upon these innovations, we present
Optimus-3, a general-purpose agent for Minecraft. Extensive experimental
results demonstrate that Optimus-3 surpasses both generalist multimodal large
language models and existing state-of-the-art agents across a wide range of
tasks in the Minecraft environment. Project page:
https://cybertronagent.github.io/Optimus-3.github.io/
Draft-based Approximate Inference for LLMs
Authors: name: Kevin Galim, name: Ethan Ewer, name: Wonjun Kang, name: Minjae Lee, name: Hyung Il Koo, name: Kangwook Lee
Published: 2025-06-10 02:37:46

Optimizing inference for long-context Large Language Models (LLMs) is
increasingly important due to the quadratic compute and linear memory
complexity of Transformers. Existing approximation methods, such as key-value
(KV) cache dropping, sparse attention, and prompt compression, typically rely
on rough predictions of token or KV pair importance. We propose a novel
framework for approximate LLM inference that leverages small draft models to
more accurately predict the importance of tokens and KV pairs. Specifically, we
introduce two instantiations of our proposed framework: (i) SpecKV, which
leverages a draft output to accurately assess the importance of each KV pair
for more effective KV cache dropping, and (ii) SpecPC, which uses the draft
model's attention activations to identify and discard unimportant prompt
tokens. To the best of our knowledge, this is the first work to use draft
models for approximate LLM inference acceleration, extending their utility
beyond traditional lossless speculative decoding. We motivate our methods with
theoretical and empirical analyses, and show a strong correlation between the
attention patterns of draft and target models. Extensive experiments on
long-context benchmarks show that our methods consistently achieve higher
accuracy than existing baselines, while preserving the same improvements in
memory usage, latency, and throughput. Our code is available at
https://github.com/furiosa-ai/draft-based-approx-llm.
MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness
  Against VLM-based Attacks
Authors: name: Zonglin Wu, name: Yule Xue, name: Xin Wei, name: Yiren Song
Published: 2025-06-06 11:02:01

As automated attack techniques rapidly advance, CAPTCHAs remain a critical
defense mechanism against malicious bots. However, existing CAPTCHA schemes
encompass a diverse range of modalities -- from static distorted text and
obfuscated images to interactive clicks, sliding puzzles, and logic-based
questions -- yet the community still lacks a unified, large-scale, multimodal
benchmark to rigorously evaluate their security robustness. To address this
gap, we introduce MCA-Bench, a comprehensive and reproducible benchmarking
suite that integrates heterogeneous CAPTCHA types into a single evaluation
protocol. Leveraging a shared vision-language model backbone, we fine-tune
specialized cracking agents for each CAPTCHA category, enabling consistent,
cross-modal assessments. Extensive experiments reveal that MCA-Bench
effectively maps the vulnerability spectrum of modern CAPTCHA designs under
varied attack settings, and crucially offers the first quantitative analysis of
how challenge complexity, interaction depth, and model solvability interrelate.
Based on these findings, we propose three actionable design principles and
identify key open challenges, laying the groundwork for systematic CAPTCHA
hardening, fair benchmarking, and broader community collaboration. Datasets and
code are available online.
Text-Aware Image Restoration with Diffusion Models
Authors: name: Jaewon Min, name: Jin Hyeon Kim, name: Paul Hyunbin Cho, name: Jaeeun Lee, name: Jihye Park, name: Minkyu Park, name: Sangpil Kim, name: Hyunhee Park, name: Seungryong Kim
Published: 2025-06-11 17:59:46

Image restoration aims to recover degraded images. However, existing
diffusion-based restoration methods, despite great success in natural image
restoration, often struggle to faithfully reconstruct textual regions in
degraded images. Those methods frequently generate plausible but incorrect
text-like patterns, a phenomenon we refer to as text-image hallucination. In
this paper, we introduce Text-Aware Image Restoration (TAIR), a novel
restoration task that requires the simultaneous recovery of visual contents and
textual fidelity. To tackle this task, we present SA-Text, a large-scale
benchmark of 100K high-quality scene images densely annotated with diverse and
complex text instances. Furthermore, we propose a multi-task diffusion
framework, called TeReDiff, that integrates internal features from diffusion
models into a text-spotting module, enabling both components to benefit from
joint training. This allows for the extraction of rich text representations,
which are utilized as prompts in subsequent denoising steps. Extensive
experiments demonstrate that our approach consistently outperforms
state-of-the-art restoration methods, achieving significant gains in text
recognition accuracy. See our project page: https://cvlab-kaist.github.io/TAIR/
Eliciting Fine-Tuned Transformer Capabilities via Inference-Time
  Techniques
Authors: name: Asankhaya Sharma
Published: 2025-06-09 08:37:19

Large language models have transformed natural language processing, yet
supervised fine-tuning (SFT) remains computationally intensive. This paper
formally proves that capabilities acquired through SFT can be approximated by a
base transformer model using inference-time techniques, specifically in-context
learning (ICL), without altering model parameters, under idealized assumptions
including unbounded computational resources and access to the fine-tuning
dataset. We extend these results to practical scenarios with finite context
lengths and partial dataset access. For text generation tasks with fixed output
length l, datasets of size Oleft( m V{varepsilon^2} log
m{delta} right) or, with bounded context, Oleft( l
log V{varepsilon^2} log 1{delta} right) suffice to approximate
fine-tuned behavior across m contexts within error varepsilon, where V
is the vocabulary size and delta is the failure probability. For linear
classification, datasets of size Oleft( d{varepsilon}
right) or, with fixed context, Oleft( 1{varepsilon^2} log
1{delta} right) are sufficient, where d is the input dimension.
Grounded in the Turing completeness of transformers, these results provide a
theoretical foundation for resource-efficient deployment of large language
models, with practical techniques like retrieval-augmented generation bridging
theory to real-world applications.
VerIF: Verification Engineering for Reinforcement Learning in
  Instruction Following
Authors: name: Hao Peng, name: Yunjia Qi, name: Xiaozhi Wang, name: Bin Xu, name: Lei Hou, name: Juanzi Li
Published: 2025-06-11 17:10:36

Reinforcement learning with verifiable rewards (RLVR) has become a key
technique for enhancing large language models (LLMs), with verification
engineering playing a central role. However, best practices for RL in
instruction following remain underexplored. In this work, we explore the
verification challenge in RL for instruction following and propose VerIF, a
verification method that combines rule-based code verification with LLM-based
verification from a large reasoning model (e.g., QwQ-32B). To support this
approach, we construct a high-quality instruction-following dataset,
VerInstruct, containing approximately 22,000 instances with associated
verification signals. We apply RL training with VerIF to two models, achieving
significant improvements across several representative instruction-following
benchmarks. The trained models reach state-of-the-art performance among models
of comparable size and generalize well to unseen constraints. We further
observe that their general capabilities remain unaffected, suggesting that RL
with VerIF can be integrated into existing RL recipes to enhance overall model
performance. We have released our datasets, codes, and models to facilitate
future research at https://github.com/THU-KEG/VerIF.
VRBench: A Benchmark for Multi-Step Reasoning in Long Narrative Videos
Authors: name: Jiashuo Yu, name: Yue Wu, name: Meng Chu, name: Zhifei Ren, name: Zizheng Huang, name: Pei Chu, name: Ruijie Zhang, name: Yinan He, name: Qirui Li, name: Songze Li, name: Zhenxiang Li, name: Zhongying Tu, name: Conghui He, name: Yu Qiao, name: Yali Wang, name: Yi Wang, name: Limin Wang
Published: 2025-06-12 16:17:17

We present VRBench, the first long narrative video benchmark crafted for
evaluating large models' multi-step reasoning capabilities, addressing
limitations in existing evaluations that overlook temporal reasoning and
procedural validity. It comprises 1,010 long videos (with an average duration
of 1.6 hours), along with 9,468 human-labeled multi-step question-answering
pairs and 30,292 reasoning steps with timestamps. These videos are curated via
a multi-stage filtering process including expert inter-rater reviewing to
prioritize plot coherence. We develop a human-AI collaborative framework that
generates coherent reasoning chains, each requiring multiple temporally
grounded steps, spanning seven types (e.g., event attribution, implicit
inference). VRBench designs a multi-phase evaluation pipeline that assesses
models at both the outcome and process levels. Apart from the MCQs for the
final results, we propose a progress-level LLM-guided scoring metric to
evaluate the quality of the reasoning chain from multiple dimensions
comprehensively. Through extensive evaluations of 12 LLMs and 16 VLMs on
VRBench, we undertake a thorough analysis and provide valuable insights that
advance the field of multi-step reasoning.
SWE-Factory: Your Automated Factory for Issue Resolution Training Data
  and Evaluation Benchmarks
Authors: name: Lianghong Guo, name: Yanlin Wang, name: Caihua Li, name: Pengyu Yang, name: Jiachi Chen, name: Wei Tao, name: Yingtian Zou, name: Duyu Tang, name: Zibin Zheng
Published: 2025-06-12 17:54:17

Constructing large-scale datasets for the GitHub issue resolution task is
crucial for both training and evaluating the software engineering capabilities
of Large Language Models (LLMs). However, the traditional process for creating
such benchmarks is notoriously challenging and labor-intensive, particularly in
the stages of setting up evaluation environments, grading test outcomes, and
validating task instances. In this paper, we propose SWE-Factory, an automated
pipeline designed to address these challenges. To tackle these issues, our
pipeline integrates three core automated components. First, we introduce
SWE-Builder, a multi-agent system that automates evaluation environment
construction, which employs four specialized agents that work in a
collaborative, iterative loop and leverages an environment memory pool to
enhance efficiency. Second, we introduce a standardized, exit-code-based
grading method that eliminates the need for manually writing custom parsers.
Finally, we automate the fail2pass validation process using these reliable exit
code signals. Experiments on 671 issues across four programming languages show
that our pipeline can effectively construct valid task instances; for example,
with GPT-4.1-mini, our SWE-Builder constructs 269 valid instances at 0.045 per
instance, while with Gemini-2.5-flash, it achieves comparable performance at
the lowest cost of 0.024 per instance. We also demonstrate that our
exit-code-based grading achieves 100% accuracy compared to manual inspection,
and our automated fail2pass validation reaches a precision of 0.92 and a recall
of 1.00. We hope our automated pipeline will accelerate the collection of
large-scale, high-quality GitHub issue resolution datasets for both training
and evaluation. Our code and datasets are released at
https://github.com/DeepSoftwareAnalytics/swe-factory.
CoRT: Code-integrated Reasoning within Thinking
Authors: name: Chengpeng Li, name: Zhengyang Tang, name: Ziniu Li, name: Mingfeng Xue, name: Keqin Bao, name: Tian Ding, name: Ruoyu Sun, name: Benyou Wang, name: Xiang Wang, name: Junyang Lin, name: Dayiheng Liu
Published: 2025-06-11 14:59:02

Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.
Efficient Part-level 3D Object Generation via Dual Volume Packing
Authors: name: Jiaxiang Tang, name: Ruijie Lu, name: Zhaoshuo Li, name: Zekun Hao, name: Xuan Li, name: Fangyin Wei, name: Shuran Song, name: Gang Zeng, name: Ming-Yu Liu, name: Tsung-Yi Lin
Published: 2025-06-11 17:55:03

Recent progress in 3D object generation has greatly improved both the quality
and efficiency. However, most existing methods generate a single mesh with all
parts fused together, which limits the ability to edit or manipulate individual
parts. A key challenge is that different objects may have a varying number of
parts. To address this, we propose a new end-to-end framework for part-level 3D
object generation. Given a single input image, our method generates
high-quality 3D objects with an arbitrary number of complete and semantically
meaningful parts. We introduce a dual volume packing strategy that organizes
all parts into two complementary volumes, allowing for the creation of complete
and interleaved parts that assemble into the final object. Experiments show
that our model achieves better quality, diversity, and generalization than
previous image-based part-level generation methods.
Can Vision Language Models Infer Human Gaze Direction? A Controlled
  Study
Authors: name: Zory Zhang, name: Pinyuan Feng, name: Bingyang Wang, name: Tianwei Zhao, name: Suyang Yu, name: Qingying Gao, name: Hokin Deng, name: Ziqiao Ma, name: Yijiang Li, name: Dezhi Luo
Published: 2025-06-04 17:59:25

Gaze-referential inference--the ability to infer what others are looking
at--is a critical component of a theory of mind that underpins natural human-AI
interaction. In a controlled study, we evaluated this skill across 111 Vision
Language Models (VLMs) using photos taken with manipulated difficulty and
variability, comparing performance with that of human participants (N = 65),
and analyzed behaviors using mixed-effects models. We found that 94 of the 111
VLMs failed to do better than random guessing, while humans achieved
near-ceiling accuracy. VLMs even respond with each choice almost equally
frequently. Are they randomly guessing? Although most VLMs struggle, when we
zoom in on five of the top-tier VLMs with above-chance performance, we find
that their performance declined with increasing task difficulty but varied only
slightly across different prompts and scene objects. These behavioral features
cannot be explained by considering them as random guessers. Instead, they
likely use a combination of heuristics and guessing such that their performance
is subject to the task difficulty but robust to perceptual variations. This
suggests that VLMs, lacking gaze inference capability, have yet to become
technologies that can naturally interact with humans, but the potential
remains.
TTT-Bench: A Benchmark for Evaluating Reasoning Ability with Simple and
  Novel Tic-Tac-Toe-style Games
Authors: name: Prakamya Mishra, name: Jiang Liu, name: Jialian Wu, name: Xiaodong Yu, name: Zicheng Liu, name: Emad Barsoum
Published: 2025-06-11 22:03:19

Large reasoning models (LRMs) have demonstrated impressive reasoning
capabilities across a broad range of tasks including Olympiad-level
mathematical problems, indicating evidence of their complex reasoning
abilities. While many reasoning benchmarks focus on the STEM domain, the
ability of LRMs to reason correctly in broader task domains remains
underexplored. In this work, we introduce TTT-Bench, a new benchmark
that is designed to evaluate basic strategic, spatial, and logical reasoning
abilities in LRMs through a suite of four two-player Tic-Tac-Toe-style games
that humans can effortlessly solve from a young age. We propose a simple yet
scalable programmatic approach for generating verifiable two-player game
problems for TTT-Bench. Although these games are trivial for humans, they
require reasoning about the intentions of the opponent, as well as the game
board's spatial configurations, to ensure a win. We evaluate a diverse set of
state-of-the-art LRMs, and discover that the models that excel at hard
math problems frequently fail at these simple reasoning games. Further testing
reveals that our evaluated reasoning models score on average downarrow 41\%
\& downarrow 5\% lower on TTT-Bench compared to MATH 500 \& AIME 2024
respectively, with larger models achieving higher performance using shorter
reasoning traces, where most of the models struggle on long-term strategic
reasoning situations on simple and new TTT-Bench tasks.
A Call for Collaborative Intelligence: Why Human-Agent Systems Should
  Precede AI Autonomy
Authors: name: Henry Peng Zou, name: Wei-Chieh Huang, name: Yaozu Wu, name: Chunyu Miao, name: Dongyuan Li, name: Aiwei Liu, name: Yue Zhou, name: Yankai Chen, name: Weizhi Zhang, name: Yangning Li, name: Liancheng Fang, name: Renhe Jiang, name: Philip S. Yu
Published: 2025-06-11 06:08:13

Recent improvements in large language models (LLMs) have led many researchers
to focus on building fully autonomous AI agents. This position paper questions
whether this approach is the right path forward, as these autonomous systems
still have problems with reliability, transparency, and understanding the
actual requirements of human. We suggest a different approach: LLM-based
Human-Agent Systems (LLM-HAS), where AI works with humans rather than replacing
them. By keeping human involved to provide guidance, answer questions, and
maintain control, these systems can be more trustworthy and adaptable. Looking
at examples from healthcare, finance, and software development, we show how
human-AI teamwork can handle complex tasks better than AI working alone. We
also discuss the challenges of building these collaborative systems and offer
practical solutions. This paper argues that progress in AI should not be
measured by how independent systems become, but by how well they can work with
humans. The most promising future for AI is not in systems that take over human
roles, but in those that enhance human capabilities through meaningful
partnership.