Orionnnn/hub-stats huggingface dataset:
LTD-Bench: Evaluating Large Language Models by Letting Them Draw
Authors: name: Liuhao Lin, name: Ke Li, name: Zihan Xu, name: Yuchen Shi, name: Yulei Qin, name: Yan Zhang, name: Xing Sun, name: Rongrong Ji
Published: 2025-11-04 08:11:23

Current evaluation paradigms for large language models (LLMs) represent a
critical blind spot in AI research--relying on opaque numerical metrics that
conceal fundamental limitations in spatial reasoning while providing no
intuitive understanding of model capabilities. This deficiency creates a
dangerous disconnect between reported performance and practical abilities,
particularly for applications requiring physical world understanding. We
introduce LTD-Bench, a breakthrough benchmark that transforms LLM evaluation
from abstract scores to directly observable visual outputs by requiring models
to generate drawings through dot matrices or executable code. This approach
makes spatial reasoning limitations immediately apparent even to non-experts,
bridging the fundamental gap between statistical performance and intuitive
assessment. LTD-Bench implements a comprehensive methodology with complementary
generation tasks (testing spatial imagination) and recognition tasks (assessing
spatial perception) across three progressively challenging difficulty levels,
methodically evaluating both directions of the critical language-spatial
mapping. Our extensive experiments with state-of-the-art models expose an
alarming capability gap: even LLMs achieving impressive results on traditional
benchmarks demonstrate profound deficiencies in establishing bidirectional
mappings between language and spatial concept--a fundamental limitation that
undermines their potential as genuine world models. Furthermore, LTD-Bench's
visual outputs enable powerful diagnostic analysis, offering a potential
approach to investigate model similarity.
Discriminately Treating Motion Components Evolves Joint Depth and
  Ego-Motion Learning
Authors: name: Mengtan Zhang, name: Zizhan Guo, name: Hongbo Zhao, name: Yi Feng, name: Zuyi Xiong, name: Yue Wang, name: Shaoyi Du, name: Hanli Wang, name: Rui Fan
Published: 2025-11-03 12:14:52

Unsupervised learning of depth and ego-motion, two fundamental 3D perception
tasks, has made significant strides in recent years. However, most methods
treat ego-motion as an auxiliary task, either mixing all motion types or
excluding depth-independent rotational motions in supervision. Such designs
limit the incorporation of strong geometric constraints, reducing reliability
and robustness under diverse conditions. This study introduces a discriminative
treatment of motion components, leveraging the geometric regularities of their
respective rigid flows to benefit both depth and ego-motion estimation. Given
consecutive video frames, network outputs first align the optical axes and
imaging planes of the source and target cameras. Optical flows between frames
are transformed through these alignments, and deviations are quantified to
impose geometric constraints individually on each ego-motion component,
enabling more targeted refinement. These alignments further reformulate the
joint learning process into coaxial and coplanar forms, where depth and each
translation component can be mutually derived through closed-form geometric
relationships, introducing complementary constraints that improve depth
robustness. DiMoDE, a general depth and ego-motion joint learning framework
incorporating these designs, achieves state-of-the-art performance on multiple
public datasets and a newly collected diverse real-world dataset, particularly
under challenging conditions. Our source code will be publicly available at
mias.group/DiMoDE upon publication.
ChartM^3: A Multi-Stage Code-Driven Pipeline for Constructing
  Multi-Dimensional and Multi-Step Visual Reasoning Data in Chart Comprehension
Authors: name: Duo Xu, name: Hao Cheng, name: Xin Lin, name: Zhen Xie, name: Hao Wang
Published: 2025-11-04 09:45:34

Complex chart understanding tasks demand advanced visual recognition and
reasoning capabilities from multimodal large language models (MLLMs). However,
current research provides limited coverage of complex chart scenarios and
computation-intensive reasoning tasks prevalent in real-world applications.
This study proposes an automated multi-stage code-driven pipeline for
systematically generating visual reasoning datasets to address these
limitations. The pipeline integrates retrieval-augmented generation (RAG) to
retrieve professional chart templates and employs chain-of-thought (CoT)
strategies to generate reasoning codes that simulate real data distributions,
thereby driving chart rendering and question-related statistical computations.
Through model-based evaluation, the pipeline enhances chart diversity and data
quality. Using this framework, we construct ChartM^3, a multi-dimensional and
multi-step dataset containing 38K charts and 142K Q&A pairs for training, along
with 2,871 high-quality evaluation samples for enabling practical performance
assessment. Supervised fine-tuning (SFT) and reinforcement learning (RL)
experiments demonstrate that our dataset significantly improves reasoning
capabilities and cross-domain generalization performance, enabling smaller
models to achieve performance comparable to larger-scale models in complex
chart comprehension.
TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System
Authors: name: Yanjie Ze, name: Siheng Zhao, name: Weizhuo Wang, name: Angjoo Kanazawa, name: Rocky Duan, name: Pieter Abbeel, name: Guanya Shi, name: Jiajun Wu, name: C. Karen Liu
Published: 2025-11-04 18:58:35

Large-scale data has driven breakthroughs in robotics, from language models
to vision-language-action models in bimanual manipulation. However, humanoid
robotics lacks equally effective data collection frameworks. Existing humanoid
teleoperation systems either use decoupled control or depend on expensive
motion capture setups. We introduce TWIST2, a portable, mocap-free humanoid
teleoperation and data collection system that preserves full whole-body control
while advancing scalability. Our system leverages PICO4U VR for obtaining
real-time whole-body human motions, with a custom 2-DoF robot neck (cost around
$250) for egocentric vision, enabling holistic human-to-humanoid control. We
demonstrate long-horizon dexterous and mobile humanoid skills and we can
collect 100 demonstrations in 15 minutes with an almost 100% success rate.
Building on this pipeline, we propose a hierarchical visuomotor policy
framework that autonomously controls the full humanoid body based on egocentric
vision. Our visuomotor policy successfully demonstrates whole-body dexterous
manipulation and dynamic kicking tasks. The entire system is fully reproducible
and open-sourced at https://yanjieze.com/TWIST2 . Our collected dataset is also
open-sourced at https://twist-data.github.io .
iFlyBot-VLA Technical Report
Authors: name: Yuan Zhang, name: Chenyu Xue, name: Wenjie Xu, name: Chao Ji, name: Jiajia wu, name: Jia Pan
Published: 2025-11-01 06:24:56

We introduce iFlyBot-VLA, a large-scale Vision-Language-Action (VLA) model
trained under a novel framework. The main contributions are listed as follows:
(1) a latent action model thoroughly trained on large-scale human and robotic
manipulation videos; (2) a dual-level action representation framework that
jointly supervises both the Vision-Language Model (VLM) and the action expert
during training; (3) a mixed training strategy that combines robot trajectory
data with general QA and spatial QA datasets, effectively enhancing the 3D
perceptual and reasoning capabilities of the VLM backbone. Specifically, the
VLM is trained to predict two complementary forms of actions: latent actions,
derived from our latent action model pretrained on cross-embodiment
manipulation data, which capture implicit high-level intentions; and structured
discrete action tokens, obtained through frequency-domain transformations of
continuous control signals, which encode explicit low-level dynamics. This dual
supervision aligns the representation spaces of language, vision, and action,
enabling the VLM to directly contribute to action generation. Experimental
results on the LIBERO Franka benchmark demonstrate the superiority of our
frame-work, while real-world evaluations further show that iFlyBot-VLA achieves
competitive success rates across diverse and challenging manipulation tasks.
Furthermore, we plan to open-source a portion of our self-constructed dataset
to support future research in the community
VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation
  Models
Authors: name: Zhicheng Zhang, name: Weicheng Wang, name: Yongjie Zhu, name: Wenyu Qin, name: Pengfei Wan, name: Di Zhang, name: Jufeng Yang
Published: 2025-11-04 16:31:09

Understanding and predicting emotion from videos has gathered significant
attention in recent studies, driven by advancements in video large language
models (VideoLLMs). While advanced methods have made progress in video emotion
analysis, the intrinsic nature of emotions poses significant challenges.
Emotions are characterized by dynamic and cues-dependent properties, making it
difficult to understand complex and evolving emotional states with reasonable
rationale. To tackle these challenges, we propose a novel affective cues-guided
reasoning framework that unifies fundamental attribute perception, expression
analysis, and high-level emotional understanding in a stage-wise manner. At the
core of our approach is a family of video emotion foundation models (VidEmo),
specifically designed for emotion reasoning and instruction-following. These
models undergo a two-stage tuning process: first, curriculum emotion learning
for injecting emotion knowledge, followed by affective-tree reinforcement
learning for emotion reasoning. Moreover, we establish a foundational data
infrastructure and introduce a emotion-centric fine-grained dataset (Emo-CFG)
consisting of 2.1M diverse instruction-based samples. Emo-CFG includes
explainable emotional question-answering, fine-grained captions, and associated
rationales, providing essential resources for advancing emotion understanding
tasks. Experimental results demonstrate that our approach achieves competitive
performance, setting a new milestone across 15 face perception tasks.
LiveSecBench: A Dynamic and Culturally-Relevant AI Safety Benchmark for
  LLMs in Chinese Context
Authors: name: Yudong Li, name: Zhongliang Yang, name: Kejiang Chen, name: Wenxuan Wang, name: Tianxin Zhang, name: Sifang Wan, name: Kecheng Wang, name: Haitian Li, name: Xu Wang, name: Lefan Cheng, name: Youdan Yang, name: Baocheng Chen, name: Ziyu Liu, name: Yufei Sun, name: Liyan Wu, name: Wenya Wen, name: Xingchi Gu, name: Peiru Yang
Published: 2025-11-04 08:44:09

In this work, we propose LiveSecBench, a dynamic and continuously updated
safety benchmark specifically for Chinese-language LLM application scenarios.
LiveSecBench evaluates models across six critical dimensions (Legality, Ethics,
Factuality, Privacy, Adversarial Robustness, and Reasoning Safety) rooted in
the Chinese legal and social frameworks. This benchmark maintains relevance
through a dynamic update schedule that incorporates new threat vectors, such as
the planned inclusion of Text-to-Image Generation Safety and Agentic Safety in
the next update. For now, LiveSecBench (v251030) has evaluated 18 LLMs,
providing a landscape of AI safety in the context of Chinese language. The
leaderboard is publicly accessible at https://livesecbench.intokentech.cn/.
VCode: a Multimodal Coding Benchmark with SVG as Symbolic Visual
  Representation
Authors: name: Kevin Qinghong Lin, name: Yuhao Zheng, name: Hangyu Ran, name: Dantong Zhu, name: Dongxing Mao, name: Linjie Li, name: Philip Torr, name: Alex Jinpeng Wang
Published: 2025-11-04 18:00:18

Code has emerged as a precise and executable medium for reasoning and action
in the agent era. Yet, progress has largely focused on language-centric tasks
such as program synthesis and debugging, leaving visual-centric coding
underexplored. Inspired by how humans reason over sketches, we advocate SVG
code as a compact, interpretable, and executable visual representation. We
introduce VCode, a benchmark that reframes multimodal understanding as code
generation: given an image, a model must produce SVG that preserves symbolic
meaning for downstream reasoning. VCode covers three domains - general
commonsense (MM-Vet), professional disciplines (MMMU), and visual-centric
perception (CV-Bench). To assess symbolic fidelity, we propose CodeVQA, a novel
evaluation protocol in which a policy model answers questions over rendered
SVGs; correct answers indicate faithful symbolic preservation. Empirically,
frontier VLMs struggle to generate faithful SVGs, revealing a persistent gap
between language-centric and visual-centric coding. To close this gap, we
introduce VCoder, an agentic framework that augments VLMs along two axes: (i)
Thinking with Revision, which iteratively analyzes discrepancies and refines
SVG code; and (ii) Acting with Visual Tools, where detectors and parsers supply
structured cues such as objects, shapes, and text beyond the model's intrinsic
capacity. Across benchmarks, frontier VLMs with strong reasoning capabilities
score well overall yet remain limited in professional knowledge and 3D
reasoning. VCoder delivers a 12.3-point overall gain over the top-performing
Claude-4-Opus. Human studies show that both humans and VLMs perform worse on
rendered SVGs, their consistency reveals the promise of symbolic visual
representation. The benchmark and code are available at
https://github.com/CSU-JPG/VCode.
D2D: Detector-to-Differentiable Critic for Improved Numeracy in
  Text-to-Image Generation
Authors: name: Nobline Yoo, name: Olga Russakovsky, name: Ye Zhu
Published: 2025-10-22 06:27:05

Text-to-image (T2I) diffusion models have achieved strong performance in
semantic alignment, yet they still struggle with generating the correct number
of objects specified in prompts. Existing approaches typically incorporate
auxiliary counting networks as external critics to enhance numeracy. However,
since these critics must provide gradient guidance during generation, they are
restricted to regression-based models that are inherently differentiable, thus
excluding detector-based models with superior counting ability, whose
count-via-enumeration nature is non-differentiable. To overcome this
limitation, we propose Detector-to-Differentiable (D2D), a novel framework that
transforms non-differentiable detection models into differentiable critics,
thereby leveraging their superior counting ability to guide numeracy
generation. Specifically, we design custom activation functions to convert
detector logits into soft binary indicators, which are then used to optimize
the noise prior at inference time with pre-trained T2I models. Our extensive
experiments on SDXL-Turbo, SD-Turbo, and Pixart-DMD across four benchmarks of
varying complexity (low-density, high-density, and multi-object scenarios)
demonstrate consistent and substantial improvements in object counting accuracy
(e.g., boosting up to 13.7% on D2D-Small, a 400-prompt, low-density benchmark),
with minimal degradation in overall image quality and computational overhead.
AthenaBench: A Dynamic Benchmark for Evaluating LLMs in Cyber Threat
  Intelligence
Authors: name: Md Tanvirul Alam, name: Dipkamal Bhusal, name: Salman Ahmad, name: Nidhi Rastogi, name: Peter Worth
Published: 2025-11-03 01:45:29

Large Language Models (LLMs) have demonstrated strong capabilities in natural
language reasoning, yet their application to Cyber Threat Intelligence (CTI)
remains limited. CTI analysis involves distilling large volumes of unstructured
reports into actionable knowledge, a process where LLMs could substantially
reduce analyst workload. CTIBench introduced a comprehensive benchmark for
evaluating LLMs across multiple CTI tasks. In this work, we extend CTIBench by
developing AthenaBench, an enhanced benchmark that includes an improved dataset
creation pipeline, duplicate removal, refined evaluation metrics, and a new
task focused on risk mitigation strategies. We evaluate twelve LLMs, including
state-of-the-art proprietary models such as GPT-5 and Gemini-2.5 Pro, alongside
seven open-source models from the LLaMA and Qwen families. While proprietary
LLMs achieve stronger results overall, their performance remains subpar on
reasoning-intensive tasks, such as threat actor attribution and risk
mitigation, with open-source models trailing even further behind. These
findings highlight fundamental limitations in the reasoning capabilities of
current LLMs and underscore the need for models explicitly tailored to CTI
workflows and automation.
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor
  for GUI Grounding
Authors: name: Shijie Zhou, name: Viet Dac Lai, name: Hao Tan, name: Jihyung Kil, name: Wanrong Zhu, name: Changyou Chen, name: Ruiyi Zhang
Published: 2025-11-02 05:34:21

Graphical user interface (GUI) grounding is a key function of computer-use
agents, which maps natural-language instructions to actionable screen regions.
Existing approaches based on Multimodal Large Language Models (MLLMs) typically
formulate it as a text-based coordinate generation task, yet directly
generating precise coordinates from visual inputs remains challenging and
computationally intensive. An intuitive way to implement GUI grounding is to
first select visual patches relevant to the instructions and then determine the
precise click location within those patches. Based on the observations that
general MLLMs have some native grounding capability, nested within their
attentions, we propose GUI-AIMA, an attention-based and coordinate-free
supervised fine-tuning framework for efficient GUI grounding. GUI-AIMA aligns
the intrinsic multimodal attention of MLLMs with patch-wise grounding signals.
These signals are calculated adaptively for diverse user instructions by
multi-head aggregation on simplified query-visual attention matrices. Besides,
its coordinate-free manner can easily integrate a plug-and-play zoom-in stage.
GUI-AIMA-3B was trained with only 85k screenshots, demonstrating exceptional
data efficiency and verifying that light training can trigger the native
grounding capability of MLLMs. It achieves state-of-the-art performance among
3B models, attaining an average accuracy of 58.6% on ScreenSpot-Pro and 62.2%
on OSWorld-G. Project page: https://github.com/sjz5202/GUI-AIMA
left|,circlearrowright,text{BUS},right|: A Large and
  Diverse Multimodal Benchmark for evaluating the ability of Vision-Language
  Models to understand Rebus Puzzles
Authors: name: Trishanu Das, name: Abhilash Nandy, name: Khush Bajaj, name: Deepiha S
Published: 2025-11-03 08:42:59

Understanding Rebus Puzzles (Rebus Puzzles use pictures, symbols, and letters
to represent words or phrases creatively) requires a variety of skills such as
image recognition, cognitive skills, commonsense reasoning, multi-step
reasoning, image-based wordplay, etc., making this a challenging task for even
current Vision-Language Models. In this paper, we present
left|,circlearrowright,text{BUS},right|, a large and diverse
benchmark of 1,333 English Rebus Puzzles containing different artistic styles
and levels of difficulty, spread across 18 categories such as food, idioms,
sports, finance, entertainment, etc. We also propose RebusDescProgICE, a
model-agnostic framework which uses a combination of an unstructured
description and code-based, structured reasoning, along with better,
reasoning-based in-context example selection, improving the performance of
Vision-Language Models on
left|,circlearrowright,text{BUS},right| by 2.1-4.1% and
20-30% using closed-source and open-source models respectively compared to
Chain-of-Thought Reasoning.
EBT-Policy: Energy Unlocks Emergent Physical Reasoning Capabilities
Authors: name: Travis Davies, name: Yiqi Huang, name: Alexi Gladstone, name: Yunxin Liu, name: Xiang Chen, name: Heng Ji, name: Huxian Liu, name: Luhui Hu
Published: 2025-10-31 15:21:05

Implicit policies parameterized by generative models, such as Diffusion
Policy, have become the standard for policy learning and Vision-Language-Action
(VLA) models in robotics. However, these approaches often suffer from high
computational cost, exposure bias, and unstable inference dynamics, which lead
to divergence under distribution shifts. Energy-Based Models (EBMs) address
these issues by learning energy landscapes end-to-end and modeling equilibrium
dynamics, offering improved robustness and reduced exposure bias. Yet, policies
parameterized by EBMs have historically struggled to scale effectively. Recent
work on Energy-Based Transformers (EBTs) demonstrates the scalability of EBMs
to high-dimensional spaces, but their potential for solving core challenges in
physically embodied models remains underexplored. We introduce a new
energy-based architecture, EBT-Policy, that solves core issues in robotic and
real-world settings. Across simulated and real-world tasks, EBT-Policy
consistently outperforms diffusion-based policies, while requiring less
training and inference computation. Remarkably, on some tasks it converges
within just two inference steps, a 50x reduction compared to Diffusion Policy's
100. Moreover, EBT-Policy exhibits emergent capabilities not seen in prior
models, such as zero-shot recovery from failed action sequences using only
behavior cloning and without explicit retry training. By leveraging its scalar
energy for uncertainty-aware inference and dynamic compute allocation,
EBT-Policy offers a promising path toward robust, generalizable robot behavior
under distribution shifts.
MR-Align: Meta-Reasoning Informed Factuality Alignment for Large
  Reasoning Models
Authors: name: Xinming Wang, name: Jian Xu, name: Bin Yu, name: Sheng Lian, name: Hongzhu Yi, name: Yi Chen, name: Yingjian Zhu, name: Boran Wang, name: Hongming Yang, name: Han Hu, name: Xu-Yao Zhang, name: Cheng-Lin Liu
Published: 2025-10-27 15:00:54

Large reasoning models (LRMs) show strong capabilities in complex reasoning,
yet their marginal gains on evidence-dependent factual questions are limited.
We find this limitation is partially attributable to a reasoning-answer hit
gap, where the model identifies the correct facts during reasoning but fails to
incorporate them into the final response, thereby reducing factual fidelity. To
address this issue, we propose MR-ALIGN, a Meta-Reasoning informed alignment
framework that enhances factuality without relying on external verifiers.
MR-ALIGN quantifies state transition probabilities along the model's thinking
process and constructs a transition-aware implicit reward that reinforces
beneficial reasoning patterns while suppressing defective ones at the atomic
thinking segments. This re-weighting reshapes token-level signals into
probability-aware segment scores, encouraging coherent reasoning trajectories
that are more conducive to factual correctness. Empirical evaluations across
four factual QA datasets and one long-form factuality benchmark show that
MR-ALIGN consistently improves accuracy and truthfulness while reducing
misleading reasoning. These results highlight that aligning the reasoning
process itself, rather than merely the outputs, is pivotal for advancing
factuality in LRMs.