Orionnnn/hub-stats huggingface dataset:
FoNE: Precise Single-Token Number Embeddings via Fourier Features
Authors: name: Tianyi Zhou, name: Deqing Fu, name: Mahdi Soltanolkotabi, name: Robin Jia, name: Vatsal Sharan
Published: 2025-02-13 19:54:59

Large Language Models (LLMs) typically represent numbers using multiple
tokens, which requires the model to aggregate these tokens to interpret
numerical values. This fragmentation makes both training and inference less
efficient and adversely affects the model's performance on number-related
tasks. Inspired by the observation that pre-trained LLMs internally learn
Fourier-like features for number tokens, we propose Fourier Number Embedding
(FoNE), a novel method that directly maps numbers into the embedding space with
their Fourier features. FoNE encodes each number as a single token with only
two embedding dimensions per digit, effectively capturing numerical values
without fragmentation. This compact representation accelerates both training
and inference. Compared to traditional subword and digit-wise embeddings, FoNE
not only reduces computational overhead but also achieves higher accuracy
across various numerical tasks including addition, subtraction and
multiplication. On 6-digit decimal addition, FoNE requires 64times less data
to achieve 99% accuracy than subword and digit-wise embeddings while using
3times and 6times fewer tokens per number, respectively. Furthermore,
FoNE is the only method that yields 100% accuracy on over 100,000 test examples
for addition, subtraction, and multiplication. The codes and visualization are
available at https://fouriernumber.github.io/.
MM-RLHF: The Next Step Forward in Multimodal LLM Alignment
Authors: name: Yi-Fan Zhang, name: Tao Yu, name: Haochen Tian, name: Chaoyou Fu, name: Peiyan Li, name: Jianshu Zeng, name: Wulin Xie, name: Yang Shi, name: Huanyu Zhang, name: Junkang Wu, name: Xue Wang, name: Yibo Hu, name: Bin Wen, name: Fan Yang, name: Zhang Zhang, name: Tingting Gao, name: Di Zhang, name: Liang Wang, name: Rong Jin, name: Tieniu Tan
Published: 2025-02-14 18:59:51

Despite notable advancements in Multimodal Large Language Models (MLLMs),
most state-of-the-art models have not undergone thorough alignment with human
preferences. This gap exists because current alignment research has primarily
achieved progress in specific areas (e.g., hallucination reduction), while the
broader question of whether aligning models with human preferences can
systematically enhance MLLM capability remains largely unexplored. To this end,
we introduce MM-RLHF, a dataset containing 120k fine-grained,
human-annotated preference comparison pairs. This dataset represents a
substantial advancement over existing resources, offering superior size,
diversity, annotation granularity, and quality. Leveraging this dataset, we
propose several key innovations to improve both the quality of reward models
and the efficiency of alignment algorithms. Notably, we introduce a
Critique-Based Reward Model, which generates critiques of model outputs before
assigning scores, offering enhanced interpretability and more informative
feedback compared to traditional scalar reward mechanisms. Additionally, we
propose Dynamic Reward Scaling, a method that adjusts the loss weight of each
sample according to the reward signal, thereby optimizing the use of
high-quality comparison pairs. Our approach is rigorously evaluated across
10 distinct dimensions and 27 benchmarks, with results
demonstrating significant and consistent improvements in model performance.
Specifically, fine-tuning LLaVA-ov-7B with MM-RLHF and our alignment algorithm
leads to a 19.5% increase in conversational abilities and a
60% improvement in safety.
  We have open-sourced the preference dataset, reward model, training and
evaluation code, as well as reward modeling and safety benchmarks. For more
details, please visit our project page: https://mm-rlhf.github.io.
Region-Adaptive Sampling for Diffusion Transformers
Authors: name: Ziming Liu, name: Yifan Yang, name: Chengruidong Zhang, name: Yiqi Zhang, name: Lili Qiu, name: Yang You, name: Yuqing Yang
Published: 2025-02-14 18:59:36

Diffusion models (DMs) have become the leading choice for generative tasks
across diverse domains. However, their reliance on multiple sequential forward
passes significantly limits real-time performance. Previous acceleration
methods have primarily focused on reducing the number of sampling steps or
reusing intermediate results, failing to leverage variations across spatial
regions within the image due to the constraints of convolutional U-Net
structures. By harnessing the flexibility of Diffusion Transformers (DiTs) in
handling variable number of tokens, we introduce RAS, a novel, training-free
sampling strategy that dynamically assigns different sampling ratios to regions
within an image based on the focus of the DiT model. Our key observation is
that during each sampling step, the model concentrates on semantically
meaningful regions, and these areas of focus exhibit strong continuity across
consecutive steps. Leveraging this insight, RAS updates only the regions
currently in focus, while other regions are updated using cached noise from the
previous step. The model's focus is determined based on the output from the
preceding step, capitalizing on the temporal consistency we observed. We
evaluate RAS on Stable Diffusion 3 and Lumina-Next-T2I, achieving speedups up
to 2.36x and 2.51x, respectively, with minimal degradation in generation
quality. Additionally, a user study reveals that RAS delivers comparable
qualities under human evaluation while achieving a 1.6x speedup. Our approach
makes a significant step towards more efficient diffusion transformers,
enhancing their potential for real-time applications.
ZeroBench: An Impossible Visual Benchmark for Contemporary Large
  Multimodal Models
Authors: name: Jonathan Roberts, name: Mohammad Reza Taesiri, name: Ansh Sharma, name: Akash Gupta, name: Samuel Roberts, name: Ioana Croitoru, name: Simion-Vlad Bogolin, name: Jialu Tang, name: Florian Langer, name: Vyas Raina, name: Vatsal Raina, name: Hanyi Xiong, name: Vishaal Udandarao, name: Jingyi Lu, name: Shiyang Chen, name: Sam Purkis, name: Tianshuo Yan, name: Wenye Lin, name: Gyungin Shin, name: Qiaochu Yang, name: Anh Totti Nguyen, name: Kai Han, name: Samuel Albanie
Published: 2025-02-13 18:59:11

Large Multimodal Models (LMMs) exhibit major shortfalls when interpreting
images and, by some measures, have poorer spatial cognition than small children
or animals. Despite this, they attain high scores on many popular visual
benchmarks, with headroom rapidly eroded by an ongoing surge of model progress.
To address this, there is a pressing need for difficult benchmarks that remain
relevant for longer. We take this idea to its limit by introducing ZeroBench-a
lightweight visual reasoning benchmark that is entirely impossible for
contemporary frontier LMMs. Our benchmark consists of 100 manually curated
questions and 334 less difficult subquestions. We evaluate 20 LMMs on
ZeroBench, all of which score 0.0%, and rigorously analyse the errors. To
encourage progress in visual understanding, we publicly release ZeroBench.
STMA: A Spatio-Temporal Memory Agent for Long-Horizon Embodied Task
  Planning
Authors: name: Mingcong Lei, name: Yiming Zhao, name: Ge Wang, name: Zhixin Mai, name: Shuguang Cui, name: Yatong Han, name: Jinke Ren
Published: 2025-02-14 14:12:09

A key objective of embodied intelligence is enabling agents to perform
long-horizon tasks in dynamic environments while maintaining robust
decision-making and adaptability. To achieve this goal, we propose the
Spatio-Temporal Memory Agent (STMA), a novel framework designed to enhance task
planning and execution by integrating spatio-temporal memory. STMA is built
upon three critical components: (1) a spatio-temporal memory module that
captures historical and environmental changes in real time, (2) a dynamic
knowledge graph that facilitates adaptive spatial reasoning, and (3) a
planner-critic mechanism that iteratively refines task strategies. We evaluate
STMA in the TextWorld environment on 32 tasks, involving multi-step planning
and exploration under varying levels of complexity. Experimental results
demonstrate that STMA achieves a 31.25% improvement in success rate and a 24.7%
increase in average score compared to the state-of-the-art model. The results
highlight the effectiveness of spatio-temporal memory in advancing the memory
capabilities of embodied agents.
Latent Radiance Fields with 3D-aware 2D Representations
Authors: name: Chaoyi Zhou, name: Xi Liu, name: Feng Luo, name: Siyu Huang
Published: 2025-02-13 18:59:09

Latent 3D reconstruction has shown great promise in empowering 3D semantic
understanding and 3D generation by distilling 2D features into the 3D space.
However, existing approaches struggle with the domain gap between 2D feature
space and 3D representations, resulting in degraded rendering performance. To
address this challenge, we propose a novel framework that integrates 3D
awareness into the 2D latent space. The framework consists of three stages: (1)
a correspondence-aware autoencoding method that enhances the 3D consistency of
2D latent representations, (2) a latent radiance field (LRF) that lifts these
3D-aware 2D representations into 3D space, and (3) a VAE-Radiance Field
(VAE-RF) alignment strategy that improves image decoding from the rendered 2D
representations. Extensive experiments demonstrate that our method outperforms
the state-of-the-art latent 3D reconstruction approaches in terms of synthesis
performance and cross-dataset generalizability across diverse indoor and
outdoor scenes. To our knowledge, this is the first work showing the radiance
field representations constructed from 2D latent representations can yield
photorealistic 3D reconstruction performance.
Mathematical Reasoning in Large Language Models: Assessing Logical and
  Arithmetic Errors across Wide Numerical Ranges
Authors: name: Safal Shrestha, name: Minwu Kim, name: Keith Ross
Published: 2025-02-12 09:53:10

Mathematical reasoning in Large Language Models (LLMs) is often evaluated
using benchmarks with limited numerical ranges, failing to reflect real-world
problem-solving across diverse scales. Furthermore, most existing evaluation
methods only compare model outputs to ground-truth answers, obscuring insights
into reasoning processes. To address these limitations, we introduce
GSM-Ranges, a dataset generator derived from GSM8K that systematically perturbs
numerical values in math problems to assess model robustness across varying
numerical scales. Additionally, we propose a novel grading methodology that
distinguishes between logical and non-logical errors, offering a more precise
evaluation of reasoning processes beyond computational accuracy. Our
experiments with various models reveal a significant increase in logical error
rates-up to 14 percentage points-as numerical complexity rises, demonstrating a
general weakness in reasoning with out-of-distribution numerical values.
Moreover, while models demonstrate high accuracy on standalone arithmetic
tasks, their performance deteriorates substantially when computations are
embedded within word problems. These findings provide a comprehensive
evaluation of LLMs' mathematical reasoning capabilities and inform future
research directions for improving numerical generalization in language models.
VFX Creator: Animated Visual Effect Generation with Controllable
  Diffusion Transformer
Authors: name: Xinyu Liu, name: Ailing Zeng, name: Wei Xue, name: Harry Yang, name: Wenhan Luo, name: Qifeng Liu, name: Yike Guo
Published: 2025-02-09 18:12:25

Crafting magic and illusions is one of the most thrilling aspects of
filmmaking, with visual effects (VFX) serving as the powerhouse behind
unforgettable cinematic experiences. While recent advances in generative
artificial intelligence have driven progress in generic image and video
synthesis, the domain of controllable VFX generation remains relatively
underexplored. In this work, we propose a novel paradigm for animated VFX
generation as image animation, where dynamic effects are generated from
user-friendly textual descriptions and static reference images.
  Our work makes two primary contributions: (i) Open-VFX, the first
high-quality VFX video dataset spanning 15 diverse effect categories, annotated
with textual descriptions, instance segmentation masks for spatial
conditioning, and start-end timestamps for temporal control. (ii) VFX Creator,
a simple yet effective controllable VFX generation framework based on a Video
Diffusion Transformer. The model incorporates a spatial and temporal
controllable LoRA adapter, requiring minimal training videos. Specifically, a
plug-and-play mask control module enables instance-level spatial manipulation,
while tokenized start-end motion timestamps embedded in the diffusion process,
alongside the text encoder, allow precise temporal control over effect timing
and pace.
  Extensive experiments on the Open-VFX test set demonstrate the superiority of
the proposed system in generating realistic and dynamic effects, achieving
state-of-the-art performance and generalization ability in both spatial and
temporal controllability. Furthermore, we introduce a specialized metric to
evaluate the precision of temporal control. By bridging traditional VFX
techniques with generative approaches, VFX Creator unlocks new possibilities
for efficient and high-quality video effect generation, making advanced VFX
accessible to a broader audience.
DexTrack: Towards Generalizable Neural Tracking Control for Dexterous
  Manipulation from Human References
Authors: name: Xueyi Liu, name: Jianibieke Adalibieke, name: Qianwei Han, name: Yuzhe Qin, name: Li Yi
Published: 2025-02-13 18:59:13

We address the challenge of developing a generalizable neural tracking
controller for dexterous manipulation from human references. This controller
aims to manage a dexterous robot hand to manipulate diverse objects for various
purposes defined by kinematic human-object interactions. Developing such a
controller is complicated by the intricate contact dynamics of dexterous
manipulation and the need for adaptivity, generalizability, and robustness.
Current reinforcement learning and trajectory optimization methods often fall
short due to their dependence on task-specific rewards or precise system
models. We introduce an approach that curates large-scale successful robot
tracking demonstrations, comprising pairs of human references and robot
actions, to train a neural controller. Utilizing a data flywheel, we
iteratively enhance the controller's performance, as well as the number and
quality of successful tracking demonstrations. We exploit available tracking
demonstrations and carefully integrate reinforcement learning and imitation
learning to boost the controller's performance in dynamic environments. At the
same time, to obtain high-quality tracking demonstrations, we individually
optimize per-trajectory tracking by leveraging the learned tracking controller
in a homotopy optimization method. The homotopy optimization, mimicking
chain-of-thought, aids in solving challenging trajectory tracking problems to
increase demonstration diversity. We showcase our success by training a
generalizable neural controller and evaluating it in both simulation and real
world. Our method achieves over a 10% improvement in success rates compared to
leading baselines. The project website with animated results is available at
https://meowuu7.github.io/DexTrack/.
3CAD: A Large-Scale Real-World 3C Product Dataset for Unsupervised
  Anomaly
Authors: name: Enquan Yang, name: Peng Xing, name: Hanyang Sun, name: Wenbo Guo, name: Yuanwei Ma, name: Zechao Li, name: Dan Zeng
Published: 2025-02-09 03:37:54

Industrial anomaly detection achieves progress thanks to datasets such as
MVTec-AD and VisA. However, they suf- fer from limitations in terms of the
number of defect sam- ples, types of defects, and availability of real-world
scenes. These constraints inhibit researchers from further exploring the
performance of industrial detection with higher accuracy. To this end, we
propose a new large-scale anomaly detection dataset called 3CAD, which is
derived from real 3C produc- tion lines. Specifically, the proposed 3CAD
includes eight different types of manufactured parts, totaling 27,039 high-
resolution images labeled with pixel-level anomalies. The key features of 3CAD
are that it covers anomalous regions of different sizes, multiple anomaly
types, and the possibility of multiple anomalous regions and multiple anomaly
types per anomaly image. This is the largest and first anomaly de- tection
dataset dedicated to 3C product quality control for community exploration and
development. Meanwhile, we in- troduce a simple yet effective framework for
unsupervised anomaly detection: a Coarse-to-Fine detection paradigm with
Recovery Guidance (CFRG). To detect small defect anoma- lies, the proposed CFRG
utilizes a coarse-to-fine detection paradigm. Specifically, we utilize a
heterogeneous distilla- tion model for coarse localization and then fine
localiza- tion through a segmentation model. In addition, to better capture
normal patterns, we introduce recovery features as guidance. Finally, we report
the results of our CFRG frame- work and popular anomaly detection methods on
the 3CAD dataset, demonstrating strong competitiveness and providing a highly
challenging benchmark to promote the development of the anomaly detection
field. Data and code are available: https://github.com/EnquanYang2022/3CAD.
Can this Model Also Recognize Dogs? Zero-Shot Model Search from Weights
Authors: name: Jonathan Kahana, name: Or Nathan, name: Eliahu Horwitz, name: Yedid Hoshen
Published: 2025-02-13 18:59:44

With the increasing numbers of publicly available models, there are probably
pretrained, online models for most tasks users require. However, current model
search methods are rudimentary, essentially a text-based search in the
documentation, thus users cannot find the relevant models. This paper presents
ProbeLog, a method for retrieving classification models that can recognize a
target concept, such as "Dog", without access to model metadata or training
data. Differently from previous probing methods, ProbeLog computes a descriptor
for each output dimension (logit) of each model, by observing its responses on
a fixed set of inputs (probes). Our method supports both logit-based retrieval
("find more logits like this") and zero-shot, text-based retrieval ("find all
logits corresponding to dogs"). As probing-based representations require
multiple costly feedforward passes through the model, we develop a method,
based on collaborative filtering, that reduces the cost of encoding
repositories by 3x. We demonstrate that ProbeLog achieves high retrieval
accuracy, both in real-world and fine-grained search tasks and is scalable to
full-size repositories.
CoSER: Coordinating LLM-Based Persona Simulation of Established Roles
Authors: name: Xintao Wang, name: Heng Wang, name: Yifei Zhang, name: Xinfeng Yuan, name: Rui Xu, name: Jen-tse Huang, name: Siyu Yuan, name: Haoran Guo, name: Jiangjie Chen, name: Wei Wang, name: Yanghua Xiao, name: Shuchang Zhou
Published: 2025-02-13 08:55:24

Role-playing language agents (RPLAs) have emerged as promising applications
of large language models (LLMs). However, simulating established characters
presents a challenging task for RPLAs, due to the lack of authentic character
datasets and nuanced evaluation methods using such data. In this paper, we
present CoSER, a collection of a high-quality dataset, open models, and an
evaluation protocol towards effective RPLAs of established characters. The
CoSER dataset covers 17,966 characters from 771 renowned books. It provides
authentic dialogues with real-world intricacies, as well as diverse data types
such as conversation setups, character experiences and internal thoughts.
Drawing from acting methodology, we introduce given-circumstance acting for
training and evaluating role-playing LLMs, where LLMs sequentially portray
multiple characters in book scenes. Using our dataset, we develop CoSER 8B and
CoSER 70B, i.e., advanced open role-playing LLMs built on LLaMA-3.1 models.
Extensive experiments demonstrate the value of the CoSER dataset for RPLA
training, evaluation and retrieval. Moreover, CoSER 70B exhibits
state-of-the-art performance surpassing or matching GPT-4o on our evaluation
and three existing benchmarks, i.e., achieving 75.80% and 93.47% accuracy on
the InCharacter and LifeChoice benchmarks respectively.
SQuARE: Sequential Question Answering Reasoning Engine for Enhanced
  Chain-of-Thought in Large Language Models
Authors: name: Daniel Fleischer, name: Moshe Berchansky, name: Gad Markovits, name: Moshe Wasserblat
Published: 2025-02-13 15:07:20

In the rapidly evolving field of Natural Language Processing, Large Language
Models (LLMs) are tasked with increasingly complex reasoning challenges.
Traditional methods like chain-of-thought prompting have shown promise but
often fall short in fully leveraging a model's reasoning capabilities. This
paper introduces SQuARE (Sequential Question Answering Reasoning Engine), a
novel prompting technique designed to improve reasoning through a
self-interrogation paradigm. Building upon CoT frameworks, SQuARE prompts
models to generate and resolve multiple auxiliary questions before tackling the
main query, promoting a more thorough exploration of various aspects of a
topic. Our expansive evaluations, conducted with Llama 3 and GPT-4o models
across multiple question-answering datasets, demonstrate that SQuARE
significantly surpasses traditional CoT prompts and existing
rephrase-and-respond methods. By systematically decomposing queries, SQuARE
advances LLM capabilities in reasoning tasks. The code is publicly available at
https://github.com/IntelLabs/RAG-FiT/tree/square.
Exploring the Potential of Encoder-free Architectures in 3D LMMs
Authors: name: Yiwen Tang, name: Zoey Guo, name: Zhuhao Wang, name: Ray Zhang, name: Qizhi Chen, name: Junli Liu, name: Delin Qu, name: Zhigang Wang, name: Dong Wang, name: Xuelong Li, name: Bin Zhao
Published: 2025-02-13 18:59:45

Encoder-free architectures have been preliminarily explored in the 2D visual
domain, yet it remains an open question whether they can be effectively applied
to 3D understanding scenarios. In this paper, we present the first
comprehensive investigation into the potential of encoder-free architectures to
overcome the challenges of encoder-based 3D Large Multimodal Models (LMMs).
These challenges include the failure to adapt to varying point cloud
resolutions and the point features from the encoder not meeting the semantic
needs of Large Language Models (LLMs). We identify key aspects for 3D LMMs to
remove the encoder and enable the LLM to assume the role of the 3D encoder: 1)
We propose the LLM-embedded Semantic Encoding strategy in the pre-training
stage, exploring the effects of various point cloud self-supervised losses. And
we present the Hybrid Semantic Loss to extract high-level semantics. 2) We
introduce the Hierarchical Geometry Aggregation strategy in the instruction
tuning stage. This incorporates inductive bias into the LLM early layers to
focus on the local details of the point clouds. To the end, we present the
first Encoder-free 3D LMM, ENEL. Our 7B model rivals the current
state-of-the-art model, ShapeLLM-13B, achieving 55.0%, 50.92%, and 42.7% on the
classification, captioning, and VQA tasks, respectively. Our results
demonstrate that the encoder-free architecture is highly promising for
replacing encoder-based architectures in the field of 3D understanding. The
code is released at https://github.com/Ivan-Tang-3D/ENEL