Orionnnn/hub-stats huggingface dataset:
OTC: Optimal Tool Calls via Reinforcement Learning
Authors: name: Hongru Wang, name: Cheng Qian, name: Wanjun Zhong, name: Xiusi Chen, name: Jiahao Qiu, name: Shijue Huang, name: Bowen Jin, name: Mengdi Wang, name: Kam-Fai Wong, name: Heng Ji
Published: 2025-04-21 05:40:05

Tool-integrated reasoning (TIR) augments large language models (LLMs) with
the ability to invoke external tools, such as search engines and code
interpreters, to solve tasks beyond the capabilities of language-only
reasoning. While reinforcement learning (RL) has shown promise in improving TIR
by optimizing final answer correctness, existing approaches often overlook the
efficiency and cost associated with tool usage. This can lead to suboptimal
behavior, including excessive tool calls that increase computational and
financial overhead, or insufficient tool use that compromises answer quality.
In this work, we propose Optimal Tool Call-controlled Policy Optimization
(OTC-PO), a simple yet effective RL-based framework that encourages models to
produce accurate answers with minimal tool calls. Our method introduces a
tool-integrated reward that jointly considers correctness and tool efficiency,
promoting high tool productivity. We instantiate this framework within both
Proximal Policy Optimization (PPO) and Group Relative Preference Optimization
(GRPO), resulting in OTC-PPO and OTC-GRPO. Experiments with Qwen-2.5 and
Qwen-Math across multiple QA benchmarks show that our approach reduces tool
calls by up to 73.1\% and improves tool productivity by up to 229.4\%, while
maintaining comparable answer accuracy. To the best of our knowledge, this is
the first RL-based framework that explicitly optimizes tool-use efficiency in
TIR.
SilVar-Med: A Speech-Driven Visual Language Model for Explainable
  Abnormality Detection in Medical Imaging
Authors: name: Tan-Hanh Pham, name: Chris Ngo, name: Trong-Duong Bui, name: Minh Luu Quang, name: Tan-Huong Pham, name: Truong-Son Hy
Published: 2025-04-14 18:51:37

Medical Visual Language Models have shown great potential in various
healthcare applications, including medical image captioning and diagnostic
assistance. However, most existing models rely on text-based instructions,
limiting their usability in real-world clinical environments especially in
scenarios such as surgery, text-based interaction is often impractical for
physicians. In addition, current medical image analysis models typically lack
comprehensive reasoning behind their predictions, which reduces their
reliability for clinical decision-making. Given that medical diagnosis errors
can have life-changing consequences, there is a critical need for interpretable
and rational medical assistance. To address these challenges, we introduce an
end-to-end speech-driven medical VLM, SilVar-Med, a multimodal medical image
assistant that integrates speech interaction with VLMs, pioneering the task of
voice-based communication for medical image analysis. In addition, we focus on
the interpretation of the reasoning behind each prediction of medical
abnormalities with a proposed reasoning dataset. Through extensive experiments,
we demonstrate a proof-of-concept study for reasoning-driven medical image
interpretation with end-to-end speech interaction. We believe this work will
advance the field of medical AI by fostering more transparent, interactive, and
clinically viable diagnostic support systems. Our code and dataset are publicly
available at SiVar-Med.
RF-DETR Object Detection vs YOLOv12 : A Study of Transformer-based and
  CNN-based Architectures for Single-Class and Multi-Class Greenfruit Detection
  in Complex Orchard Environments Under Label Ambiguity
Authors: name: Ranjan Sapkota, name: Rahul Harsha Cheppally, name: Ajay Sharda, name: Manoj Karkee
Published: 2025-04-17 17:08:11

This study conducts a detailed comparison of RF-DETR object detection base
model and YOLOv12 object detection model configurations for detecting
greenfruits in a complex orchard environment marked by label ambiguity,
occlusions, and background blending. A custom dataset was developed featuring
both single-class (greenfruit) and multi-class (occluded and non-occluded
greenfruits) annotations to assess model performance under dynamic real-world
conditions. RF-DETR object detection model, utilizing a DINOv2 backbone and
deformable attention, excelled in global context modeling, effectively
identifying partially occluded or ambiguous greenfruits. In contrast, YOLOv12
leveraged CNN-based attention for enhanced local feature extraction, optimizing
it for computational efficiency and edge deployment. RF-DETR achieved the
highest mean Average Precision (mAP50) of 0.9464 in single-class detection,
proving its superior ability to localize greenfruits in cluttered scenes.
Although YOLOv12N recorded the highest mAP@50:95 of 0.7620, RF-DETR
consistently outperformed in complex spatial scenarios. For multi-class
detection, RF-DETR led with an mAP@50 of 0.8298, showing its capability to
differentiate between occluded and non-occluded fruits, while YOLOv12L scored
highest in mAP@50:95 with 0.6622, indicating better classification in detailed
occlusion contexts. Training dynamics analysis highlighted RF-DETR's swift
convergence, particularly in single-class settings where it plateaued within 10
epochs, demonstrating the efficiency of transformer-based architectures in
adapting to dynamic visual data. These findings validate RF-DETR's
effectiveness for precision agricultural applications, with YOLOv12 suited for
fast-response scenarios. >Index Terms: RF-DETR object detection, YOLOv12,
YOLOv13, YOLOv14, YOLOv15, YOLOE, YOLO World, YOLO, You Only Look Once,
Roboflow, Detection Transformers, CNNs
CoMotion: Concurrent Multi-person 3D Motion
Authors: name: Alejandro Newell, name: Peiyun Hu, name: Lahav Lipson, name: Stephan R. Richter, name: Vladlen Koltun
Published: 2025-04-16 15:40:15

We introduce an approach for detecting and tracking detailed 3D poses of
multiple people from a single monocular camera stream. Our system maintains
temporally coherent predictions in crowded scenes filled with difficult poses
and occlusions. Our model performs both strong per-frame detection and a
learned pose update to track people from frame to frame. Rather than match
detections across time, poses are updated directly from a new input image,
which enables online tracking through occlusion. We train on numerous image and
video datasets leveraging pseudo-labeled annotations to produce a model that
matches state-of-the-art systems in 3D pose estimation accuracy while being
faster and more accurate in tracking multiple people through time. Code and
weights are provided at https://github.com/apple/ml-comotion
Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls
  for Video Generation
Authors: name: Chenjie Cao, name: Jingkai Zhou, name: Shikai Li, name: Jingyun Liang, name: Chaohui Yu, name: Fan Wang, name: Xiangyang Xue, name: Yanwei Fu
Published: 2025-04-21 07:10:41

Camera and human motion controls have been extensively studied for video
generation, but existing approaches typically address them separately,
suffering from limited data with high-quality annotations for both aspects. To
overcome this, we present Uni3C, a unified 3D-enhanced framework for precise
control of both camera and human motion in video generation. Uni3C includes two
key contributions. First, we propose a plug-and-play control module trained
with a frozen video generative backbone, PCDController, which utilizes
unprojected point clouds from monocular depth to achieve accurate camera
control. By leveraging the strong 3D priors of point clouds and the powerful
capacities of video foundational models, PCDController shows impressive
generalization, performing well regardless of whether the inference backbone is
frozen or fine-tuned. This flexibility enables different modules of Uni3C to be
trained in specific domains, i.e., either camera control or human motion
control, reducing the dependency on jointly annotated data. Second, we propose
a jointly aligned 3D world guidance for the inference phase that seamlessly
integrates both scenic point clouds and SMPL-X characters to unify the control
signals for camera and human motion, respectively. Extensive experiments
confirm that PCDController enjoys strong robustness in driving camera motion
for fine-tuned backbones of video generation. Uni3C substantially outperforms
competitors in both camera controllability and human motion quality.
Additionally, we collect tailored validation sets featuring challenging camera
movements and human actions to validate the effectiveness of our method.
An LMM for Efficient Video Understanding via Reinforced Compression of
  Video Cubes
Authors: name: Ji Qi, name: Yuan Yao, name: Yushi Bai, name: Bin Xu, name: Juanzi Li, name: Zhiyuan Liu, name: Tat-Seng Chua
Published: 2025-04-21 17:57:21

Large Multimodal Models (LMMs) uniformly perceive video frames, creating
computational inefficiency for videos with inherently varying temporal
information density. This paper present Quicksviewer, an LMM with new
perceiving paradigm that partitions a video of nonuniform density into varying
cubes using Gumbel Softmax, followed by a unified resampling for each cube to
achieve efficient video understanding. This simple and intuitive approach
dynamically compress video online based on its temporal density, significantly
reducing spatiotemporal redundancy (overall 45times compression rate), while
enabling efficient training with large receptive field. We train the model from
a language backbone through three progressive stages, each incorporating
lengthy videos on average of 420s/1fps thanks to the perceiving efficiency.
With only 0.8M total video-text samples for training, our model outperforms the
direct baseline employing a fixed partitioning strategy by a maximum of 8.72 in
accuracy, demonstrating the effectiveness in performance. On Video-MME,
Quicksviewer achieves SOTA under modest sequence lengths using just up to 5\%
of tokens per frame required by baselines. With this paradigm, scaling up the
number of input frames reveals a clear power law of the model capabilities. It
is also empirically verified that the segments generated by the cubing network
can help for analyzing continuous events in videos.
Eagle 2.5: Boosting Long-Context Post-Training for Frontier
  Vision-Language Models
Authors: name: Guo Chen, name: Zhiqi Li, name: Shihao Wang, name: Jindong Jiang, name: Yicheng Liu, name: Lidong Lu, name: De-An Huang, name: Wonmin Byeon, name: Matthieu Le, name: Tuomas Rintamaki, name: Tyler Poon, name: Max Ehrlich, name: Tuomas Rintamaki, name: Tyler Poon, name: Tong Lu, name: Limin Wang, name: Bryan Catanzaro, name: Jan Kautz, name: Andrew Tao, name: Zhiding Yu, name: Guilin Liu
Published: 2025-04-21 17:57:28

We introduce Eagle 2.5, a family of frontier vision-language models (VLMs)
for long-context multimodal learning. Our work addresses the challenges in long
video comprehension and high-resolution image understanding, introducing a
generalist framework for both tasks. The proposed training framework
incorporates Automatic Degrade Sampling and Image Area Preservation, two
techniques that preserve contextual integrity and visual details. The framework
also includes numerous efficiency optimizations in the pipeline for
long-context data training. Finally, we propose Eagle-Video-110K, a novel
dataset that integrates both story-level and clip-level annotations,
facilitating long-video understanding. Eagle 2.5 demonstrates substantial
improvements on long-context multimodal benchmarks, providing a robust solution
to the limitations of existing VLMs. Notably, our best model Eagle 2.5-8B
achieves 72.4% on Video-MME with 512 input frames, matching the results of
top-tier commercial model such as GPT-4o and large-scale open-source models
like Qwen2.5-VL-72B and InternVL2.5-78B.
EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language
  Models
Authors: name: Ziwen Xu, name: Shuxun Wang, name: Kewei Xu, name: Haoming Xu, name: Mengru Wang, name: Xinle Deng, name: Yunzhi Yao, name: Guozhou Zheng, name: Huajun Chen, name: Ningyu Zhang
Published: 2025-04-21 14:33:55

In this paper, we introduce EasyEdit2, a framework designed to enable
plug-and-play adjustability for controlling Large Language Model (LLM)
behaviors. EasyEdit2 supports a wide range of test-time interventions,
including safety, sentiment, personality, reasoning patterns, factuality, and
language features. Unlike its predecessor, EasyEdit2 features a new
architecture specifically designed for seamless model steering. It comprises
key modules such as the steering vector generator and the steering vector
applier, which enable automatic generation and application of steering vectors
to influence the model's behavior without modifying its parameters. One of the
main advantages of EasyEdit2 is its ease of use-users do not need extensive
technical knowledge. With just a single example, they can effectively guide and
adjust the model's responses, making precise control both accessible and
efficient. Empirically, we report model steering performance across different
LLMs, demonstrating the effectiveness of these techniques. We have released the
source code on GitHub at https://github.com/zjunlp/EasyEdit along with a
demonstration notebook. In addition, we provide a demo video at
https://zjunlp.github.io/project/EasyEdit2/video for a quick introduction.
LookingGlass: Generative Anamorphoses via Laplacian Pyramid Warping
Authors: name: Pascal Chang, name: Sergio Sancho, name: Jingwei Tang, name: Markus Gross, name: Vinicius C. Azevedo
Published: 2025-04-11 18:12:01

Anamorphosis refers to a category of images that are intentionally distorted,
making them unrecognizable when viewed directly. Their true form only reveals
itself when seen from a specific viewpoint, which can be through some
catadioptric device like a mirror or a lens. While the construction of these
mathematical devices can be traced back to as early as the 17th century, they
are only interpretable when viewed from a specific vantage point and tend to
lose meaning when seen normally. In this paper, we revisit these famous optical
illusions with a generative twist. With the help of latent rectified flow
models, we propose a method to create anamorphic images that still retain a
valid interpretation when viewed directly. To this end, we introduce Laplacian
Pyramid Warping, a frequency-aware image warping technique key to generating
high-quality visuals. Our work extends Visual Anagrams (arXiv:2311.17919) to
latent space models and to a wider range of spatial transforms, enabling the
creation of novel generative perceptual illusions.
StyleMe3D: Stylization with Disentangled Priors by Multiple Encoders on
  3D Gaussians
Authors: name: Cailin Zhuang, name: Yaoqi Hu, name: Xuanyang Zhang, name: Wei Cheng, name: Jiacheng Bao, name: Shengqi Liu, name: Yiying Yang, name: Xianfang Zeng, name: Gang Yu, name: Ming Li
Published: 2025-04-21 17:59:55

3D Gaussian Splatting (3DGS) excels in photorealistic scene reconstruction
but struggles with stylized scenarios (e.g., cartoons, games) due to fragmented
textures, semantic misalignment, and limited adaptability to abstract
aesthetics. We propose StyleMe3D, a holistic framework for 3D GS style transfer
that integrates multi-modal style conditioning, multi-level semantic alignment,
and perceptual quality enhancement. Our key insights include: (1) optimizing
only RGB attributes preserves geometric integrity during stylization; (2)
disentangling low-, medium-, and high-level semantics is critical for coherent
style transfer; (3) scalability across isolated objects and complex scenes is
essential for practical deployment. StyleMe3D introduces four novel components:
Dynamic Style Score Distillation (DSSD), leveraging Stable Diffusion's latent
space for semantic alignment; Contrastive Style Descriptor (CSD) for localized,
content-aware texture transfer; Simultaneously Optimized Scale (SOS) to
decouple style details and structural coherence; and 3D Gaussian Quality
Assessment (3DG-QA), a differentiable aesthetic prior trained on human-rated
data to suppress artifacts and enhance visual harmony. Evaluated on NeRF
synthetic dataset (objects) and tandt db (scenes) datasets, StyleMe3D
outperforms state-of-the-art methods in preserving geometric details (e.g.,
carvings on sculptures) and ensuring stylistic consistency across scenes (e.g.,
coherent lighting in landscapes), while maintaining real-time rendering. This
work bridges photorealistic 3D GS and artistic stylization, unlocking
applications in gaming, virtual worlds, and digital art.
TAPIP3D: Tracking Any Point in Persistent 3D Geometry
Authors: name: Bowei Zhang, name: Lei Ke, name: Adam W. Harley, name: Katerina Fragkiadaki
Published: 2025-04-20 19:09:43

We introduce TAPIP3D, a novel approach for long-term 3D point tracking in
monocular RGB and RGB-D videos. TAPIP3D represents videos as camera-stabilized
spatio-temporal feature clouds, leveraging depth and camera motion information
to lift 2D video features into a 3D world space where camera motion is
effectively canceled. TAPIP3D iteratively refines multi-frame 3D motion
estimates within this stabilized representation, enabling robust tracking over
extended periods. To manage the inherent irregularities of 3D point
distributions, we propose a Local Pair Attention mechanism. This 3D
contextualization strategy effectively exploits spatial relationships in 3D,
forming informative feature neighborhoods for precise 3D trajectory estimation.
Our 3D-centric approach significantly outperforms existing 3D point tracking
methods and even enhances 2D tracking accuracy compared to conventional 2D
pixel trackers when accurate depth is available. It supports inference in both
camera coordinates (i.e., unstabilized) and world coordinates, and our results
demonstrate that compensating for camera motion improves tracking performance.
Our approach replaces the conventional 2D square correlation neighborhoods used
in prior 2D and 3D trackers, leading to more robust and accurate results across
various 3D point tracking benchmarks. Project Page: https://tapip3d.github.io
InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to
  Deliberative Reasoners
Authors: name: Yuhang Liu, name: Pengxiang Li, name: Congkai Xie, name: Xavier Hu, name: Xiaotian Han, name: Shengyu Zhang, name: Hongxia Yang, name: Fei Wu
Published: 2025-04-19 09:25:55

Multimodal Large Language Models (MLLMs) have powered Graphical User
Interface (GUI) Agents, showing promise in automating tasks on computing
devices. Recent works have begun exploring reasoning in GUI tasks with
encouraging results. However, many current approaches rely on manually designed
reasoning templates, which may result in reasoning that is not sufficiently
robust and adaptive for complex GUI environments. Meanwhile, some existing
agents continue to operate as Reactive Actors, relying primarily on implicit
reasoning that may lack sufficient depth for GUI tasks demanding planning and
error recovery. We argue that advancing these agents requires a shift from
reactive acting towards acting based on deliberate reasoning. To facilitate
this transformation, we introduce InfiGUI-R1, an MLLM-based GUI agent developed
through our Actor2Reasoner framework, a reasoning-centric, two-stage training
approach designed to progressively evolve agents from Reactive Actors to
Deliberative Reasoners. The first stage, Reasoning Injection, focuses on
establishing a basic reasoner. We employ Spatial Reasoning Distillation to
transfer cross-modal spatial reasoning capabilities from teacher models to
MLLMs through trajectories with explicit reasoning steps, enabling models to
integrate GUI visual-spatial information with logical reasoning before action
generation. The second stage, Deliberation Enhancement, refines the basic
reasoner into a deliberative one using Reinforcement Learning. This stage
introduces two approaches: Sub-goal Guidance, which rewards models for
generating accurate intermediate sub-goals, and Error Recovery Scenario
Construction, which creates failure-and-recovery training scenarios from
identified prone-to-error steps. Experimental results show InfiGUI-R1 achieves
strong performance in GUI grounding and trajectory tasks. Resources at
https://github.com/Reallm-Labs/InfiGUI-R1.
FlowReasoner: Reinforcing Query-Level Meta-Agents
Authors: name: Hongcheng Gao, name: Yue Liu, name: Yufei He, name: Longxu Dou, name: Chao Du, name: Zhijie Deng, name: Bryan Hooi, name: Min Lin, name: Tianyu Pang
Published: 2025-04-21 17:35:42

This paper proposes a query-level meta-agent named FlowReasoner to automate
the design of query-level multi-agent systems, i.e., one system per user query.
Our core idea is to incentivize a reasoning-based meta-agent via external
execution feedback. Concretely, by distilling DeepSeek R1, we first endow the
basic reasoning ability regarding the generation of multi-agent systems to
FlowReasoner. Then, we further enhance it via reinforcement learning (RL) with
external execution feedback. A multi-purpose reward is designed to guide the RL
training from aspects of performance, complexity, and efficiency. In this
manner, FlowReasoner is enabled to generate a personalized multi-agent system
for each user query via deliberative reasoning. Experiments on both engineering
and competition code benchmarks demonstrate the superiority of FlowReasoner.
Remarkably, it surpasses o1-mini by 10.52% accuracy across three benchmarks.
The code is available at https://github.com/sail-sg/FlowReasoner.
DRAGON: Distributional Rewards Optimize Diffusion Generative Models
Authors: name: Yatong Bai, name: Jonah Casebeer, name: Somayeh Sojoudi, name: Nicholas J. Bryan
Published: 2025-04-21 16:41:40

We present Distributional RewArds for Generative OptimizatioN (DRAGON), a
versatile framework for fine-tuning media generation models towards a desired
outcome. Compared with traditional reinforcement learning with human feedback
(RLHF) or pairwise preference approaches such as direct preference optimization
(DPO), DRAGON is more flexible. It can optimize reward functions that evaluate
either individual examples or distributions of them, making it compatible with
a broad spectrum of instance-wise, instance-to-distribution, and
distribution-to-distribution rewards. Leveraging this versatility, we construct
novel reward functions by selecting an encoder and a set of reference examples
to create an exemplar distribution. When cross-modality encoders such as CLAP
are used, the reference examples may be of a different modality (e.g., text
versus audio). Then, DRAGON gathers online and on-policy generations, scores
them to construct a positive demonstration set and a negative set, and
leverages the contrast between the two sets to maximize the reward. For
evaluation, we fine-tune an audio-domain text-to-music diffusion model with 20
different reward functions, including a custom music aesthetics model, CLAP
score, Vendi diversity, and Frechet audio distance (FAD). We further compare
instance-wise (per-song) and full-dataset FAD settings while ablating multiple
FAD encoders and reference sets. Over all 20 target rewards, DRAGON achieves an
81.45% average win rate. Moreover, reward functions based on exemplar sets
indeed enhance generations and are comparable to model-based rewards. With an
appropriate exemplar set, DRAGON achieves a 60.95% human-voted music quality
win rate without training on human preference annotations. As such, DRAGON
exhibits a new approach to designing and optimizing reward functions for
improving human-perceived quality. Sound examples at
https://ml-dragon.github.io/web.