Orionnnn/hub-stats huggingface dataset:
FLAG-Trader: Fusion LLM-Agent with Gradient-based Reinforcement Learning
  for Financial Trading
Authors: name: Guojun Xiong, name: Zhiyang Deng, name: Keyi Wang, name: Yupeng Cao, name: Haohang Li, name: Yangyang Yu, name: Xueqing Peng, name: Mingquan Lin, name: Kaleb E Smith, name: Xiao-Yang Liu, name: Jimin Huang, name: Sophia Ananiadou, name: Qianqian Xie
Published: 2025-02-17 04:45:53

Large language models (LLMs) fine-tuned on multimodal financial data have
demonstrated impressive reasoning capabilities in various financial tasks.
However, they often struggle with multi-step, goal-oriented scenarios in
interactive financial markets, such as trading, where complex agentic
approaches are required to improve decision-making. To address this, we propose
FLAG-Trader, a unified architecture integrating linguistic processing
(via LLMs) with gradient-driven reinforcement learning (RL) policy
optimization, in which a partially fine-tuned LLM acts as the policy network,
leveraging pre-trained knowledge while adapting to the financial domain through
parameter-efficient fine-tuning. Through policy gradient optimization driven by
trading rewards, our framework not only enhances LLM performance in trading but
also improves results on other financial-domain tasks. We present extensive
empirical evidence to validate these enhancements.
Rethinking Diverse Human Preference Learning through Principal Component
  Analysis
Authors: name: Feng Luo, name: Rui Yang, name: Hao Sun, name: Chunyuan Deng, name: Jiarui Yao, name: Jingyan Shen, name: Huan Zhang, name: Hanjie Chen
Published: 2025-02-18 18:55:26

Understanding human preferences is crucial for improving foundation models
and building personalized AI systems. However, preferences are inherently
diverse and complex, making it difficult for traditional reward models to
capture their full range. While fine-grained preference data can help,
collecting it is expensive and hard to scale. In this paper, we introduce
Decomposed Reward Models (DRMs), a novel approach that extracts diverse human
preferences from binary comparisons without requiring fine-grained annotations.
Our key insight is to represent human preferences as vectors and analyze them
using Principal Component Analysis (PCA). By constructing a dataset of
embedding differences between preferred and rejected responses, DRMs identify
orthogonal basis vectors that capture distinct aspects of preference. These
decomposed rewards can be flexibly combined to align with different user needs,
offering an interpretable and scalable alternative to traditional reward
models. We demonstrate that DRMs effectively extract meaningful preference
dimensions (e.g., helpfulness, safety, humor) and adapt to new users without
additional training. Our results highlight DRMs as a powerful framework for
personalized and interpretable LLM alignment.
HeadInfer: Memory-Efficient LLM Inference by Head-wise Offloading
Authors: name: Cheng Luo, name: Zefan Cai, name: Hanshi Sun, name: Jinqi Xiao, name: Bo Yuan, name: Wen Xiao, name: Junjie Hu, name: Jiawei Zhao, name: Beidi Chen, name: Anima Anandkumar
Published: 2025-02-18 06:26:05

Transformer-based large language models (LLMs) demonstrate impressive
performance in long context generation. Extending the context length has
disproportionately shifted the memory footprint of LLMs during inference to the
key-value cache (KV cache). In this paper, we propose HEADINFER, which offloads
the KV cache to CPU RAM while avoiding the need to fully store the KV cache for
any transformer layer on the GPU. HEADINFER employs a fine-grained, head-wise
offloading strategy, maintaining only selective attention heads KV cache on the
GPU while computing attention output dynamically. Through roofline analysis, we
demonstrate that HEADINFER maintains computational efficiency while
significantly reducing memory footprint. We evaluate HEADINFER on the
Llama-3-8B model with a 1-million-token sequence, reducing the GPU memory
footprint of the KV cache from 128 GB to 1 GB and the total GPU memory usage
from 207 GB to 17 GB, achieving a 92% reduction compared to BF16 baseline
inference. Notably, HEADINFER enables 4-million-token inference with an 8B
model on a single consumer GPU with 24GB memory (e.g., NVIDIA RTX 4090) without
approximation methods.
Phantom: Subject-consistent video generation via cross-modal alignment
Authors: name: Lijie Liu, name: Tianxiang Ma, name: Bingchuan Li, name: Zhuowei Chen, name: Jiawei Liu, name: Qian He, name: Xinglong Wu
Published: 2025-02-16 11:02:50

The continuous development of foundational models for video generation is
evolving into various applications, with subject-consistent video generation
still in the exploratory stage. We refer to this as Subject-to-Video, which
extracts subject elements from reference images and generates
subject-consistent video through textual instructions. We believe that the
essence of subject-to-video lies in balancing the dual-modal prompts of text
and image, thereby deeply and simultaneously aligning both text and visual
content. To this end, we propose Phantom, a unified video generation framework
for both single and multi-subject references. Building on existing
text-to-video and image-to-video architectures, we redesign the joint
text-image injection model and drive it to learn cross-modal alignment via
text-image-video triplet data. In particular, we emphasize subject consistency
in human generation, covering existing ID-preserving video generation while
offering enhanced advantages. The project homepage is here
https://phantom-video.github.io/Phantom/.
Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for
  LLM-as-a-Judge
Authors: name: Qiyuan Zhang, name: Yufei Wang, name: Yuxin Jiang, name: Liangyou Li, name: Chuhan Wu, name: Yasheng Wang, name: Xin Jiang, name: Lifeng Shang, name: Ruiming Tang, name: Fuyuan Lyu, name: Chen Ma
Published: 2025-02-18 03:31:06

LLM-as-a-Judge, which generates chain-of-thought (CoT) judgments, has become
a widely adopted auto-evaluation method. However, its reliability is
compromised by the CoT reasoning's inability to capture comprehensive and
deeper details, often leading to incomplete outcomes. Existing methods mainly
rely on majority voting or criteria expansion, which is insufficient to address
the limitation in CoT. We propose Crowd-based Comparative Evaluation, which
introduces additional crowd responses to compare with the candidate responses,
thereby exposing deeper and more comprehensive details within the candidate
responses. This process effectively guides LLM-as-a-Judge to provide a more
detailed CoT judgment. Extensive experiments demonstrate that our approach
enhances evaluation reliability, achieving an average accuracy gain of 6.7%
across five benchmarks. Moreover, our method produces higher-quality CoTs that
facilitate judge distillation and exhibit superior performance in rejection
sampling for supervised fine-tuning (SFT), referred to as crowd rejection
sampling, thereby enabling more efficient SFT. Our analysis confirms that CoTs
generated by ours are more comprehensive and of higher quality, and evaluation
accuracy improves as inference scales.
RealSyn: An Effective and Scalable Multimodal Interleaved Document
  Transformation Paradigm
Authors: name: Tiancheng Gu, name: Kaicheng Yang, name: Chaoyi Zhang, name: Yin Xie, name: Xiang An, name: Ziyong Feng, name: Dongnan Liu, name: Weidong Cai, name: Jiankang Deng
Published: 2025-02-18 03:58:38

After pre-training on extensive image-text pairs, Contrastive Language-Image
Pre-training (CLIP) demonstrates promising performance on a wide variety of
benchmarks. However, a substantial volume of non-paired data, such as
multimodal interleaved documents, remains underutilized for vision-language
representation learning. To fully leverage these unpaired documents, we
initially establish a Real-World Data Extraction pipeline to extract
high-quality images and texts. Then we design a hierarchical retrieval method
to efficiently associate each image with multiple semantically relevant
realistic texts. To further enhance fine-grained visual information, we propose
an image semantic augmented generation module for synthetic text production.
Furthermore, we employ a semantic balance sampling strategy to improve dataset
diversity, enabling better learning of long-tail concepts. Based on these
innovations, we construct RealSyn, a dataset combining realistic and synthetic
texts, available in three scales: 15M, 30M, and 100M. Extensive experiments
demonstrate that RealSyn effectively advances vision-language representation
learning and exhibits strong scalability. Models pre-trained on RealSyn achieve
state-of-the-art performance on multiple downstream tasks. To facilitate future
research, the RealSyn dataset and pre-trained model weights are released at
https://github.com/deepglint/RealSyn.
SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and
  Object Manipulation
Authors: name: Zekun Qi, name: Wenyao Zhang, name: Yufei Ding, name: Runpei Dong, name: Xinqiang Yu, name: Jingwen Li, name: Lingyun Xu, name: Baoyu Li, name: Xialin He, name: Guofan Fan, name: Jiazhao Zhang, name: Jiawei He, name: Jiayuan Gu, name: Xin Jin, name: Kaisheng Ma, name: Zhizheng Zhang, name: He Wang, name: Li Yi
Published: 2025-02-18 18:59:02

Spatial intelligence is a critical component of embodied AI, promoting robots
to understand and interact with their environments. While recent advances have
enhanced the ability of VLMs to perceive object locations and positional
relationships, they still lack the capability to precisely understand object
orientations-a key requirement for tasks involving fine-grained manipulations.
Addressing this limitation not only requires geometric reasoning but also an
expressive and intuitive way to represent orientation. In this context, we
propose that natural language offers a more flexible representation space than
canonical frames, making it particularly suitable for instruction-following
robotic systems. In this paper, we introduce the concept of semantic
orientation, which defines object orientations using natural language in a
reference-frame-free manner (e.g., the ''plug-in'' direction of a USB or the
''handle'' direction of a knife). To support this, we construct OrienText300K,
a large-scale dataset of 3D models annotated with semantic orientations that
link geometric understanding to functional semantics. By integrating semantic
orientation into a VLM system, we enable robots to generate manipulation
actions with both positional and orientational constraints. Extensive
experiments in simulation and real world demonstrate that our approach
significantly enhances robotic manipulation capabilities, e.g., 48.7% accuracy
on Open6DOR and 74.9% accuracy on SIMPLER.
Sailor2: Sailing in South-East Asia with Inclusive Multilingual LLMs
Authors: name: Longxu Dou, name: Qian Liu, name: Fan Zhou, name: Changyu Chen, name: Zili Wang, name: Ziqi Jin, name: Zichen Liu, name: Tongyao Zhu, name: Cunxiao Du, name: Penghui Yang, name: Haonan Wang, name: Jiaheng Liu, name: Yongchi Zhao, name: Xiachong Feng, name: Xin Mao, name: Man Tsung Yeung, name: Kunat Pipatanakul, name: Fajri Koto, name: Min Si Thu, name: Hynek Kydlíček, name: Zeyi Liu, name: Qunshu Lin, name: Sittipong Sripaisarnmongkol, name: Kridtaphad Sae-Khow, name: Nirattisai Thongchim, name: Taechawat Konkaew, name: Narong Borijindargoon, name: Anh Dao, name: Matichon Maneegard, name: Phakphum Artkaew, name: Zheng-Xin Yong, name: Quan Nguyen, name: Wannaphong Phatthiyaphaibun, name: Hoang H. Tran, name: Mike Zhang, name: Shiqi Chen, name: Tianyu Pang, name: Chao Du, name: Xinyi Wan, name: Wei Lu, name: Min Lin
Published: 2025-02-18 16:04:57

Sailor2 is a family of cutting-edge multilingual language models for
South-East Asian (SEA) languages, available in 1B, 8B, and 20B sizes to suit
diverse applications. Building on Qwen2.5, Sailor2 undergoes continuous
pre-training on 500B tokens (400B SEA-specific and 100B replay tokens) to
support 13 SEA languages while retaining proficiency in Chinese and English.
Sailor2-20B model achieves a 50-50 win rate against GPT-4o across SEA
languages. We also deliver a comprehensive cookbook on how to develop the
multilingual model in an efficient manner, including five key aspects: data
curation, pre-training, post-training, model customization and evaluation. We
hope that Sailor2 model (Apache 2.0 license) will drive language development in
the SEA region, and Sailor2 cookbook will inspire researchers to build more
inclusive LLMs for other under-served languages.
ExaGPT: Example-Based Machine-Generated Text Detection for Human
  Interpretability
Authors: name: Ryuto Koike, name: Masahiro Kaneko, name: Ayana Niwa, name: Preslav Nakov, name: Naoaki Okazaki
Published: 2025-02-17 01:15:07

Detecting texts generated by Large Language Models (LLMs) could cause grave
mistakes due to incorrect decisions, such as undermining student's academic
dignity. LLM text detection thus needs to ensure the interpretability of the
decision, which can help users judge how reliably correct its prediction is.
When humans verify whether a text is human-written or LLM-generated, they
intuitively investigate with which of them it shares more similar spans.
However, existing interpretable detectors are not aligned with the human
decision-making process and fail to offer evidence that users easily
understand. To bridge this gap, we introduce ExaGPT, an interpretable detection
approach grounded in the human decision-making process for verifying the origin
of a text. ExaGPT identifies a text by checking whether it shares more similar
spans with human-written vs. with LLM-generated texts from a datastore. This
approach can provide similar span examples that contribute to the decision for
each span in the text as evidence. Our human evaluation demonstrates that
providing similar span examples contributes more effectively to judging the
correctness of the decision than existing interpretable methods. Moreover,
extensive experiments in four domains and three generators show that ExaGPT
massively outperforms prior powerful detectors by up to +40.9 points of
accuracy at a false positive rate of 1%.
Diffusion Models without Classifier-free Guidance
Authors: name: Zhicong Tang, name: Jianmin Bao, name: Dong Chen, name: Baining Guo
Published: 2025-02-17 18:59:50

This paper presents Model-guidance (MG), a novel objective for training
diffusion model that addresses and removes of the commonly used Classifier-free
guidance (CFG). Our innovative approach transcends the standard modeling of
solely data distribution to incorporating the posterior probability of
conditions. The proposed technique originates from the idea of CFG and is easy
yet effective, making it a plug-and-play module for existing models. Our method
significantly accelerates the training process, doubles the inference speed,
and achieve exceptional quality that parallel and even surpass concurrent
diffusion models with CFG. Extensive experiments demonstrate the effectiveness,
efficiency, scalability on different models and datasets. Finally, we establish
state-of-the-art performance on ImageNet 256 benchmarks with an FID of 1.34.
Our code is available at https://github.com/tzco/Diffusion-wo-CFG.
EQ-VAE: Equivariance Regularized Latent Space for Improved Generative
  Image Modeling
Authors: name: Theodoros Kouzelis, name: Ioannis Kakogeorgiou, name: Spyros Gidaris, name: Nikos Komodakis
Published: 2025-02-13 17:21:51

Latent generative models have emerged as a leading approach for high-quality
image synthesis. These models rely on an autoencoder to compress images into a
latent space, followed by a generative model to learn the latent distribution.
We identify that existing autoencoders lack equivariance to semantic-preserving
transformations like scaling and rotation, resulting in complex latent spaces
that hinder generative performance. To address this, we propose EQ-VAE, a
simple regularization approach that enforces equivariance in the latent space,
reducing its complexity without degrading reconstruction quality. By finetuning
pre-trained autoencoders with EQ-VAE, we enhance the performance of several
state-of-the-art generative models, including DiT, SiT, REPA and MaskGIT,
achieving a 7 speedup on DiT-XL/2 with only five epochs of SD-VAE fine-tuning.
EQ-VAE is compatible with both continuous and discrete autoencoders, thus
offering a versatile enhancement for a wide range of latent generative models.
Project page and code: https://eq-vae.github.io/.
Ask in Any Modality: A Comprehensive Survey on Multimodal
  Retrieval-Augmented Generation
Authors: name: Mohammad Mahdi Abootorabi, name: Amirhosein Zobeiri, name: Mahdi Dehghani, name: Mohammadali Mohammadkhani, name: Bardia Mohammadi, name: Omid Ghahroodi, name: Mahdieh Soleymani Baghshah, name: Ehsaneddin Asgari
Published: 2025-02-12 22:33:41

Large Language Models (LLMs) struggle with hallucinations and outdated
knowledge due to their reliance on static training data. Retrieval-Augmented
Generation (RAG) mitigates these issues by integrating external dynamic
information enhancing factual and updated grounding. Recent advances in
multimodal learning have led to the development of Multimodal RAG,
incorporating multiple modalities such as text, images, audio, and video to
enhance the generated outputs. However, cross-modal alignment and reasoning
introduce unique challenges to Multimodal RAG, distinguishing it from
traditional unimodal RAG. This survey offers a structured and comprehensive
analysis of Multimodal RAG systems, covering datasets, metrics, benchmarks,
evaluation, methodologies, and innovations in retrieval, fusion, augmentation,
and generation. We precisely review training strategies, robustness
enhancements, and loss functions, while also exploring the diverse Multimodal
RAG scenarios. Furthermore, we discuss open challenges and future research
directions to support advancements in this evolving field. This survey lays the
foundation for developing more capable and reliable AI systems that effectively
leverage multimodal dynamic external knowledge bases. Resources are available
at https://github.com/llm-lab-org/Multimodal-RAG-Survey.
IHEval: Evaluating Language Models on Following the Instruction
  Hierarchy
Authors: name: Zhihan Zhang, name: Shiyang Li, name: Zixuan Zhang, name: Xin Liu, name: Haoming Jiang, name: Xianfeng Tang, name: Yifan Gao, name: Zheng Li, name: Haodong Wang, name: Zhaoxuan Tan, name: Yichuan Li, name: Qingyu Yin, name: Bing Yin, name: Meng Jiang
Published: 2025-02-12 19:35:28

The instruction hierarchy, which establishes a priority order from system
messages to user messages, conversation history, and tool outputs, is essential
for ensuring consistent and safe behavior in language models (LMs). Despite its
importance, this topic receives limited attention, and there is a lack of
comprehensive benchmarks for evaluating models' ability to follow the
instruction hierarchy. We bridge this gap by introducing IHEval, a novel
benchmark comprising 3,538 examples across nine tasks, covering cases where
instructions in different priorities either align or conflict. Our evaluation
of popular LMs highlights their struggle to recognize instruction priorities.
All evaluated models experience a sharp performance decline when facing
conflicting instructions, compared to their original instruction-following
performance. Moreover, the most competitive open-source model only achieves 48%
accuracy in resolving such conflicts. Our results underscore the need for
targeted optimization in the future development of LMs.
Data Valuation using Neural Networks for Efficient Instruction
  Fine-Tuning
Authors: name: Ishika Agarwal, name: Dilek Hakkani-Tür
Published: 2025-02-14 07:55:47

Influence functions provide crucial insights into model training, but
existing methods suffer from large computational costs and limited
generalization. Particularly, recent works have proposed various metrics and
algorithms to calculate the influence of data using language models, which do
not scale well with large models and datasets. This is because of the expensive
forward and backward passes required for computation, substantial memory
requirements to store large models, and poor generalization of influence
estimates to new data. In this paper, we explore the use of small neural
networks -- which we refer to as the InfluenceNetwork -- to estimate influence
values, achieving up to 99% cost reduction. Our evaluation demonstrates that
influence values can be estimated with models just 0.0027% the size of full
language models (we use 7B and 8B versions). We apply our algorithm of
estimating influence values (called NN-CIFT: Neural Networks for effiCient
Instruction Fine-Tuning) to the downstream task of subset selection for general
instruction fine-tuning. In our study, we include four state-of-the-art
influence functions and show no compromise in performance, despite large
speedups, between NN-CIFT and the original influence functions. We provide an
in-depth hyperparameter analyses of NN-CIFT. The code for our method can be
found here: https://github.com/agarwalishika/NN-CIFT.
Explorer: Scaling Exploration-driven Web Trajectory Synthesis for
  Multimodal Web Agents
Authors: name: Vardaan Pahuja, name: Yadong Lu, name: Corby Rosset, name: Boyu Gou, name: Arindam Mitra, name: Spencer Whitehead, name: Yu Su, name: Ahmed Awadallah
Published: 2025-02-17 02:13:48

Recent success in large multimodal models (LMMs) has sparked promising
applications of agents capable of autonomously completing complex web tasks.
While open-source LMM agents have made significant advances in offline
evaluation benchmarks, their performance still falls substantially short of
human-level capabilities in more realistic online settings. A key bottleneck is
the lack of diverse and large-scale trajectory-level datasets across various
domains, which are expensive to collect. In this paper, we address this
challenge by developing a scalable recipe to synthesize the largest and most
diverse trajectory-level dataset to date, containing over 94K successful
multimodal web trajectories, spanning 49K unique URLs, 720K screenshots, and
33M web elements. In particular, we leverage extensive web exploration and
refinement to obtain diverse task intents. The average cost is 28 cents per
successful trajectory, making it affordable to a wide range of users in the
community. Leveraging this dataset, we train Explorer, a multimodal web agent,
and demonstrate strong performance on both offline and online web agent
benchmarks such as Mind2Web-Live, Multimodal-Mind2Web, and MiniWob++.
Additionally, our experiments highlight data scaling as a key driver for
improving web agent capabilities. We hope this study makes state-of-the-art
LMM-based agent research at a larger scale more accessible.