Orionnnn/hub-stats huggingface dataset:
EgoNormia: Benchmarking Physical Social Norm Understanding
Authors: name: MohammadHossein Rezaei, name: Yicheng Fu, name: Phil Cuvin, name: Caleb Ziems, name: Yanzhe Zhang, name: Hao Zhu, name: Diyi Yang
Published: 2025-02-27 19:54:16

Human activity is moderated by norms. When performing actions in the real
world, humans not only follow norms, but also consider the trade-off between
different norms However, machines are often trained without explicit
supervision on norm understanding and reasoning, especially when the norms are
grounded in a physical and social context. To improve and evaluate the
normative reasoning capability of vision-language models (VLMs), we present
EgoNormia |epsilon|, consisting of 1,853 ego-centric videos of human
interactions, each of which has two related questions evaluating both the
prediction and justification of normative actions. The normative actions
encompass seven categories: safety, privacy, proxemics, politeness,
cooperation, coordination/proactivity, and communication/legibility. To compile
this dataset at scale, we propose a novel pipeline leveraging video sampling,
automatic answer generation, filtering, and human validation. Our work
demonstrates that current state-of-the-art vision-language models lack robust
norm understanding, scoring a maximum of 45% on EgoNormia (versus a human bench
of 92%). Our analysis of performance in each dimension highlights the
significant risks of safety, privacy, and the lack of collaboration and
communication capability when applied to real-world agents. We additionally
show that through a retrieval-based generation method, it is possible to use
EgoNomia to enhance normative reasoning in VLMs.
How far can we go with ImageNet for Text-to-Image generation?
Authors: name: L. Degeorge, name: A. Ghosh, name: N. Dufour, name: D. Picard, name: V. Kalogeiton
Published: 2025-02-28 18:59:42

Recent text-to-image (T2I) generation models have achieved remarkable results
by training on billion-scale datasets, following a `bigger is better' paradigm
that prioritizes data quantity over quality. We challenge this established
paradigm by demonstrating that strategic data augmentation of small,
well-curated datasets can match or outperform models trained on massive
web-scraped collections. Using only ImageNet enhanced with well-designed text
and image augmentations, we achieve a +2 overall score over SD-XL on GenEval
and +5 on DPGBench while using just 1/10th the parameters and 1/1000th the
training images. Our results suggest that strategic data augmentation, rather
than massive datasets, could offer a more sustainable path forward for T2I
generation.
DexGraspVLA: A Vision-Language-Action Framework Towards General
  Dexterous Grasping
Authors: name: Yifan Zhong, name: Xuchuan Huang, name: Ruochong Li, name: Ceyao Zhang, name: Yitao Liang, name: Yaodong Yang, name: Yuanpei Chen
Published: 2025-02-28 09:57:20

Dexterous grasping remains a fundamental yet challenging problem in robotics.
A general-purpose robot must be capable of grasping diverse objects in
arbitrary scenarios. However, existing research typically relies on specific
assumptions, such as single-object settings or limited environments, leading to
constrained generalization. Our solution is DexGraspVLA, a hierarchical
framework that utilizes a pre-trained Vision-Language model as the high-level
task planner and learns a diffusion-based policy as the low-level Action
controller. The key insight lies in iteratively transforming diverse language
and visual inputs into domain-invariant representations, where imitation
learning can be effectively applied due to the alleviation of domain shift.
Thus, it enables robust generalization across a wide range of real-world
scenarios. Notably, our method achieves a 90+% success rate under thousands of
unseen object, lighting, and background combinations in a ``zero-shot''
environment. Empirical analysis further confirms the consistency of internal
model behavior across environmental variations, thereby validating our design
and explaining its generalization performance. We hope our work can be a step
forward in achieving general dexterous grasping. Our demo and code can be found
at https://dexgraspvla.github.io/.
TeleRAG: Efficient Retrieval-Augmented Generation Inference with
  Lookahead Retrieval
Authors: name: Chien-Yu Lin, name: Keisuke Kamahori, name: Yiyu Liu, name: Xiaoxiang Shi, name: Madhav Kashyap, name: Yile Gu, name: Rulin Shao, name: Zihao Ye, name: Kan Zhu, name: Stephanie Wang, name: Arvind Krishnamurthy, name: Rohan Kadekodi, name: Luis Ceze, name: Baris Kasikci
Published: 2025-02-28 11:32:22

Retrieval-augmented generation (RAG) extends large language models (LLMs)
with external data sources to enhance factual correctness and domain coverage.
Modern RAG pipelines rely on large datastores, leading to system challenges in
latency-sensitive deployments, especially when limited GPU memory is available.
To address these challenges, we propose TeleRAG, an efficient inference system
that reduces RAG latency with minimal GPU memory requirements. The core
innovation of TeleRAG is lookahead retrieval, a prefetching mechanism that
anticipates required data and transfers it from CPU to GPU in parallel with LLM
generation. By leveraging the modularity of RAG pipelines, the inverted file
index (IVF) search algorithm and similarities between queries, TeleRAG
optimally overlaps data movement and computation. Experimental results show
that TeleRAG reduces end-to-end RAG inference latency by up to 1.72x on average
compared to state-of-the-art systems, enabling faster, more memory-efficient
deployments of advanced RAG applications.
MIGE: A Unified Framework for Multimodal Instruction-Based Image
  Generation and Editing
Authors: name: Xueyun Tian, name: Wei Li, name: Bingbing Xu, name: Yige Yuan, name: Yuanzhuo Wang, name: Huawei Shen
Published: 2025-02-28 18:21:08

Despite significant progress in diffusion-based image generation,
subject-driven generation and instruction-based editing remain challenging.
Existing methods typically treat them separately, struggling with limited
high-quality data and poor generalization. However, both tasks require
capturing complex visual variations while maintaining consistency between
inputs and outputs. Therefore, we propose MIGE, a unified framework that
standardizes task representations using multimodal instructions. It treats
subject-driven generation as creation on a blank canvas and instruction-based
editing as modification of an existing image, establishing a shared
input-output formulation. MIGE introduces a novel multimodal encoder that maps
free-form multimodal instructions into a unified vision-language space,
integrating visual and semantic features through a feature fusion
mechanism.This unification enables joint training of both tasks, providing two
key advantages: (1) Cross-Task Enhancement: By leveraging shared visual and
semantic representations, joint training improves instruction adherence and
visual consistency in both subject-driven generation and instruction-based
editing. (2) Generalization: Learning in a unified format facilitates
cross-task knowledge transfer, enabling MIGE to generalize to novel
compositional tasks, including instruction-based subject-driven editing.
Experiments show that MIGE excels in both subject-driven generation and
instruction-based editing while setting a state-of-the-art in the new task of
instruction-based subject-driven editing. Code and model have been publicly
available at https://github.com/Eureka-Maggie/MIGE.
LettuceDetect: A Hallucination Detection Framework for RAG Applications
Authors: name: Ádám Kovács, name: Gábor Recski
Published: 2025-02-24 13:11:47

Retrieval Augmented Generation (RAG) systems remain vulnerable to
hallucinated answers despite incorporating external knowledge sources. We
present LettuceDetect a framework that addresses two critical limitations in
existing hallucination detection methods: (1) the context window constraints of
traditional encoder-based methods, and (2) the computational inefficiency of
LLM based approaches. Building on ModernBERT's extended context capabilities
(up to 8k tokens) and trained on the RAGTruth benchmark dataset, our approach
outperforms all previous encoder-based models and most prompt-based models,
while being approximately 30 times smaller than the best models. LettuceDetect
is a token-classification model that processes context-question-answer triples,
allowing for the identification of unsupported claims at the token level.
Evaluations on the RAGTruth corpus demonstrate an F1 score of 79.22% for
example-level detection, which is a 14.8% improvement over Luna, the previous
state-of-the-art encoder-based architecture. Additionally, the system can
process 30 to 60 examples per second on a single GPU, making it more practical
for real-world RAG applications.
Optimal Brain Apoptosis
Authors: name: Mingyuan Sun, name: Zheng Fang, name: Jiaxu Wang, name: Junjie Jiang, name: Delei Kong, name: Chenming Hu, name: Yuetong Fang, name: Renjing Xu
Published: 2025-02-25 08:03:04

The increasing complexity and parameter count of Convolutional Neural
Networks (CNNs) and Transformers pose challenges in terms of computational
efficiency and resource demands. Pruning has been identified as an effective
strategy to address these challenges by removing redundant elements such as
neurons, channels, or connections, thereby enhancing computational efficiency
without heavily compromising performance. This paper builds on the foundational
work of Optimal Brain Damage (OBD) by advancing the methodology of parameter
importance estimation using the Hessian matrix. Unlike previous approaches that
rely on approximations, we introduce Optimal Brain Apoptosis (OBA), a novel
pruning method that calculates the Hessian-vector product value directly for
each parameter. By decomposing the Hessian matrix across network layers and
identifying conditions under which inter-layer Hessian submatrices are
non-zero, we propose a highly efficient technique for computing the
second-order Taylor expansion of parameters. This approach allows for a more
precise pruning process, particularly in the context of CNNs and Transformers,
as validated in our experiments including VGG19, ResNet32, ResNet50, and
ViT-B/16 on CIFAR10, CIFAR100 and Imagenet datasets. Our code is available at
https://github.com/NEU-REAL/OBA.
Tell me why: Visual foundation models as self-explainable classifiers
Authors: name: Hugues Turbé, name: Mina Bjelogrlic, name: Gianmarco Mengaldo, name: Christian Lovis
Published: 2025-02-26 21:40:30

Visual foundation models (VFMs) have become increasingly popular due to their
state-of-the-art performance. However, interpretability remains crucial for
critical applications. In this sense, self-explainable models (SEM) aim to
provide interpretable classifiers that decompose predictions into a weighted
sum of interpretable concepts. Despite their promise, recent studies have shown
that these explanations often lack faithfulness. In this work, we combine VFMs
with a novel prototypical architecture and specialized training objectives. By
training only a lightweight head (approximately 1M parameters) on top of frozen
VFMs, our approach (ProtoFM) offers an efficient and interpretable solution.
Evaluations demonstrate that our approach achieves competitive classification
performance while outperforming existing models across a range of
interpretability metrics derived from the literature. Code is available at
https://github.com/hturbe/proto-fm.
Chain of Draft: Thinking Faster by Writing Less
Authors: name: Silei Xu, name: Wenhao Xie, name: Lingxiao Zhao, name: Pengcheng He
Published: 2025-02-25 19:36:06

Large Language Models (LLMs) have demonstrated remarkable performance in
solving complex reasoning tasks through mechanisms like Chain-of-Thought (CoT)
prompting, which emphasizes verbose, step-by-step reasoning. However, humans
typically employ a more efficient strategy: drafting concise intermediate
thoughts that capture only essential information. In this work, we propose
Chain of Draft (CoD), a novel paradigm inspired by human cognitive processes,
where LLMs generate minimalistic yet informative intermediate reasoning outputs
while solving tasks. By reducing verbosity and focusing on critical insights,
CoD matches or surpasses CoT in accuracy while using as little as only 7.6% of
the tokens, significantly reducing cost and latency across various reasoning
tasks.
ViDoRAG: Visual Document Retrieval-Augmented Generation via Dynamic
  Iterative Reasoning Agents
Authors: name: Qiuchen Wang, name: Ruixue Ding, name: Zehui Chen, name: Weiqi Wu, name: Shihang Wang, name: Pengjun Xie, name: Feng Zhao
Published: 2025-02-25 09:26:12

Understanding information from visually rich documents remains a significant
challenge for traditional Retrieval-Augmented Generation (RAG) methods.
Existing benchmarks predominantly focus on image-based question answering (QA),
overlooking the fundamental challenges of efficient retrieval, comprehension,
and reasoning within dense visual documents. To bridge this gap, we introduce
ViDoSeek, a novel dataset designed to evaluate RAG performance on visually rich
documents requiring complex reasoning. Based on it, we identify key limitations
in current RAG approaches: (i) purely visual retrieval methods struggle to
effectively integrate both textual and visual features, and (ii) previous
approaches often allocate insufficient reasoning tokens, limiting their
effectiveness. To address these challenges, we propose ViDoRAG, a novel
multi-agent RAG framework tailored for complex reasoning across visual
documents. ViDoRAG employs a Gaussian Mixture Model (GMM)-based hybrid strategy
to effectively handle multi-modal retrieval. To further elicit the model's
reasoning capabilities, we introduce an iterative agent workflow incorporating
exploration, summarization, and reflection, providing a framework for
investigating test-time scaling in RAG domains. Extensive experiments on
ViDoSeek validate the effectiveness and generalization of our approach.
Notably, ViDoRAG outperforms existing methods by over 10% on the competitive
ViDoSeek benchmark.
Sim-to-Real Reinforcement Learning for Vision-Based Dexterous
  Manipulation on Humanoids
Authors: name: Toru Lin, name: Kartik Sachdev, name: Linxi Fan, name: Jitendra Malik, name: Yuke Zhu
Published: 2025-02-27 18:59:52

Reinforcement learning has delivered promising results in achieving human- or
even superhuman-level capabilities across diverse problem domains, but success
in dexterous robot manipulation remains limited. This work investigates the key
challenges in applying reinforcement learning to solve a collection of
contact-rich manipulation tasks on a humanoid embodiment. We introduce novel
techniques to overcome the identified challenges with empirical validation. Our
main contributions include an automated real-to-sim tuning module that brings
the simulated environment closer to the real world, a generalized reward design
scheme that simplifies reward engineering for long-horizon contact-rich
manipulation tasks, a divide-and-conquer distillation process that improves the
sample efficiency of hard-exploration problems while maintaining sim-to-real
performance, and a mixture of sparse and dense object representations to bridge
the sim-to-real perception gap. We show promising results on three humanoid
dexterous manipulation tasks, with ablation studies on each technique. Our work
presents a successful approach to learning humanoid dexterous manipulation
using sim-to-real reinforcement learning, achieving robust generalization and
high performance without the need for human demonstration.
HAIC: Improving Human Action Understanding and Generation with Better
  Captions for Multi-modal Large Language Models
Authors: name: Xiao Wang, name: Jingyun Hua, name: Weihong Lin, name: Yuanxing Zhang, name: Fuzheng Zhang, name: Jianlong Wu, name: Di Zhang, name: Liqiang Nie
Published: 2025-02-28 07:53:40

Recent Multi-modal Large Language Models (MLLMs) have made great progress in
video understanding. However, their performance on videos involving human
actions is still limited by the lack of high-quality data. To address this, we
introduce a two-stage data annotation pipeline. First, we design strategies to
accumulate videos featuring clear human actions from the Internet. Second,
videos are annotated in a standardized caption format that uses human
attributes to distinguish individuals and chronologically details their actions
and interactions. Through this pipeline, we curate two datasets, namely
HAICTrain and HAICBench. HAICTrain comprises 126K video-caption pairs
generated by Gemini-Pro and verified for training purposes. Meanwhile,
HAICBench includes 500 manually annotated video-caption pairs and
1,400 QA pairs, for a comprehensive evaluation of human action understanding.
Experimental results demonstrate that training with HAICTrain not only
significantly enhances human understanding abilities across 4 benchmarks, but
can also improve text-to-video generation results. Both the HAICTrain and
HAICBench are released at https://huggingface.co/datasets/KuaishouHAIC/HAIC.
SoS1: O1 and R1-Like Reasoning LLMs are Sum-of-Square Solvers
Authors: name: Kechen Li, name: Wenqi Zhu, name: Coralia Cartis, name: Tianbo Ji, name: Shiwei Liu
Published: 2025-02-27 21:41:43

Large Language Models (LLMs) have achieved human-level proficiency across
diverse tasks, but their ability to perform rigorous mathematical problem
solving remains an open challenge. In this work, we investigate a fundamental
yet computationally intractable problem: determining whether a given
multivariate polynomial is nonnegative. This problem, closely related to
Hilbert's Seventeenth Problem, plays a crucial role in global polynomial
optimization and has applications in various fields. First, we introduce
SoS-1K, a meticulously curated dataset of approximately 1,000 polynomials,
along with expert-designed reasoning instructions based on five progressively
challenging criteria. Evaluating multiple state-of-the-art LLMs, we find that
without structured guidance, all models perform only slightly above the random
guess baseline 50%. However, high-quality reasoning instructions significantly
improve accuracy, boosting performance up to 81%. Furthermore, our 7B model,
SoS-7B, fine-tuned on SoS-1K for just 4 hours, outperforms the 671B DeepSeek-V3
and GPT-4o-mini in accuracy while only requiring 1.8% and 5% of the computation
time needed for letters, respectively. Our findings highlight the potential of
LLMs to push the boundaries of mathematical reasoning and tackle NP-hard
problems.
LiteASR: Efficient Automatic Speech Recognition with Low-Rank
  Approximation
Authors: name: Keisuke Kamahori, name: Jungo Kasai, name: Noriyuki Kojima, name: Baris Kasikci
Published: 2025-02-27 22:52:21

Modern automatic speech recognition (ASR) models, such as OpenAI's Whisper,
rely on deep encoder-decoder architectures, and their encoders are a critical
bottleneck for efficient deployment due to high computational intensity. We
introduce LiteASR, a low-rank compression scheme for ASR encoders that
significantly reduces inference costs while maintaining transcription accuracy.
Our approach leverages the strong low-rank properties observed in intermediate
activations: by applying principal component analysis (PCA) with a small
calibration dataset, we approximate linear transformations with a chain of
low-rank matrix multiplications, and further optimize self-attention to work in
the reduced dimension. Evaluation results show that our method can compress
Whisper large-v3's encoder size by over 50%, matching Whisper medium's size
with better transcription accuracy, thereby establishing a new Pareto-optimal
frontier of efficiency and performance. The code of LiteASR is available at
https://github.com/efeslab/LiteASR.
DeepSolution: Boosting Complex Engineering Solution Design via
  Tree-based Exploration and Bi-point Thinking
Authors: name: Zhuoqun Li, name: Haiyang Yu, name: Xuanang Chen, name: Hongyu Lin, name: Yaojie Lu, name: Fei Huang, name: Xianpei Han, name: Yongbin Li, name: Le Sun
Published: 2025-02-28 05:23:10

Designing solutions for complex engineering challenges is crucial in human
production activities. However, previous research in the retrieval-augmented
generation (RAG) field has not sufficiently addressed tasks related to the
design of complex engineering solutions. To fill this gap, we introduce a new
benchmark, SolutionBench, to evaluate a system's ability to generate complete
and feasible solutions for engineering problems with multiple complex
constraints. To further advance the design of complex engineering solutions, we
propose a novel system, SolutionRAG, that leverages the tree-based exploration
and bi-point thinking mechanism to generate reliable solutions. Extensive
experimental results demonstrate that SolutionRAG achieves state-of-the-art
(SOTA) performance on the SolutionBench, highlighting its potential to enhance
the automation and reliability of complex engineering solution design in
real-world applications.
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning
  Trajectories for Complex Problem Solving
Authors: name: Mihir Parmar, name: Xin Liu, name: Palash Goyal, name: Yanfei Chen, name: Long Le, name: Swaroop Mishra, name: Hossein Mobahi, name: Jindong Gu, name: Zifeng Wang, name: Hootan Nakhost, name: Chitta Baral, name: Chen-Yu Lee, name: Tomas Pfister, name: Hamid Palangi
Published: 2025-02-22 06:21:56

Recent agent frameworks and inference-time algorithms often struggle with
complex planning problems due to limitations in verifying generated plans or
reasoning and varying complexity of instances within a single task. Many
existing methods for these tasks either perform task-level verification without
considering constraints or apply inference-time algorithms without adapting to
instance-level complexity. To address these limitations, we propose PlanGEN, a
model-agnostic and easily scalable agent framework with three key components:
constraint, verification, and selection agents. Specifically, our approach
proposes constraint-guided iterative verification to enhance performance of
inference-time algorithms--Best of N, Tree-of-Thought, and REBASE. In PlanGEN
framework, the selection agent optimizes algorithm choice based on instance
complexity, ensuring better adaptability to complex planning problems.
Experimental results demonstrate significant improvements over the strongest
baseline across multiple benchmarks, achieving state-of-the-art results on
NATURAL PLAN (sim8%uparrow), OlympiadBench (sim4%uparrow), DocFinQA
(sim7%uparrow), and GPQA (sim1%uparrow). Our key finding highlights
that constraint-guided iterative verification improves inference-time
algorithms, and adaptive selection further boosts performance on complex
planning and reasoning problems.