Orionnnn/hub-stats huggingface dataset:
StockBench: Can LLM Agents Trade Stocks Profitably In Real-world
  Markets?
Authors: name: Yanxu Chen, name: Zijun Yao, name: Yantao Liu, name: Jin Ye, name: Jianing Yu, name: Lei Hou, name: Juanzi Li
Published: 2025-10-02 16:54:57

Large language models (LLMs) have recently demonstrated strong capabilities
as autonomous agents, showing promise in reasoning, tool use, and sequential
decision-making. While prior benchmarks have evaluated LLM agents in domains
such as software engineering and scientific discovery, the finance domain
remains underexplored, despite its direct relevance to economic value and
high-stakes decision-making. Existing financial benchmarks primarily test
static knowledge through question answering, but they fall short of capturing
the dynamic and iterative nature of trading. To address this gap, we introduce
StockBench, a contamination-free benchmark designed to evaluate LLM agents in
realistic, multi-month stock trading environments. Agents receive daily market
signals -- including prices, fundamentals, and news -- and must make sequential
buy, sell, or hold decisions. Performance is assessed using financial metrics
such as cumulative return, maximum drawdown, and the Sortino ratio. Our
evaluation of state-of-the-art proprietary (e.g., GPT-5, Claude-4) and
open-weight (e.g., Qwen3, Kimi-K2, GLM-4.5) models shows that while most LLM
agents struggle to outperform the simple buy-and-hold baseline, several models
demonstrate the potential to deliver higher returns and manage risk more
effectively. These findings highlight both the challenges and opportunities in
developing LLM-powered financial agents, showing that excelling at static
financial knowledge tasks does not necessarily translate into successful
trading strategies. We release StockBench as an open-source resource to support
reproducibility and advance future research in this domain.
LongCodeZip: Compress Long Context for Code Language Models
Authors: name: Yuling Shi, name: Yichun Qian, name: Hongyu Zhang, name: Beijun Shen, name: Xiaodong Gu
Published: 2025-10-01 02:54:57

Code generation under long contexts is becoming increasingly critical as
Large Language Models (LLMs) are required to reason over extensive information
in the codebase. While recent advances enable code LLMs to process long inputs,
high API costs and generation latency remain substantial bottlenecks. Existing
context pruning techniques, such as LLMLingua, achieve promising results for
general text but overlook code-specific structures and dependencies, leading to
suboptimal performance in programming tasks. In this paper, we propose
LongCodeZip, a novel plug-and-play code compression framework designed
specifically for code LLMs. LongCodeZip employs a dual-stage strategy: (1)
coarse-grained compression, which identifies and ranks function-level chunks
using conditional perplexity with respect to the instruction, retaining only
the most relevant functions; and (2) fine-grained compression, which segments
retained functions into blocks based on perplexity and selects an optimal
subset under an adaptive token budget to maximize relevance. Evaluations across
multiple tasks, including code completion, summarization, and question
answering, show that LongCodeZip consistently outperforms baseline methods,
achieving up to a 5.6x compression ratio without degrading task performance. By
effectively reducing context size while preserving essential information,
LongCodeZip enables LLMs to better scale to real-world, large-scale code
scenarios, advancing the efficiency and capability of code intelligence
applications.
SKYLENAGE Technical Report: Mathematical Reasoning and
  Contest-Innovation Benchmarks for Multi-Level Math Evaluation
Authors: name: Hu Wei, name: Ze Xu, name: Boyu Yang, name: Linlin Miao, name: Weiqi Zhai, name: Yihan Li, name: Zixuan Li, name: Zhijun Wang, name: Boya Wang, name: Jianwei Yu, name: Jialing Yuan, name: Xiaoyue Zhang, name: Cheng He, name: Minglei Chen, name: Zifan Zhang, name: Qianhui Li, name: Wei Wang, name: Xiang Xu
Published: 2025-09-24 02:09:32

Large language models (LLMs) now perform strongly on many public math suites,
yet frontier separation within mathematics increasingly suffers from ceiling
effects. We present two complementary benchmarks: SKYLENAGE-ReasoningMATH, a
100-item, structure-aware diagnostic set with per-item metadata on length,
numeric density, and symbolic complexity; and SKYLENAGE-MATH, a 150-item
contest-style suite spanning four stages from high school to doctoral under a
seven-subject taxonomy. We evaluate fifteen contemporary LLM variants under a
single setup and analyze subject x model and grade x model performance. On the
contest suite, the strongest model reaches 44% while the runner-up reaches 37%;
accuracy declines from high school to doctoral, and top systems exhibit a
doctoral-to-high-school retention near 79%. On the reasoning set, the best
model attains 81% overall, and hardest-slice results reveal clear robustness
gaps between leaders and the mid-tier. In summary, we release
SKYLENAGE-ReasoningMATH and report aggregate results for SKYLENAGE-MATH;
together, SKYLENAGE provides a hard, reasoning-centered and broadly covering
math benchmark with calibrated difficulty and rich metadata, serving as a
reference benchmark for future evaluations of mathematical reasoning.
VOGUE: Guiding Exploration with Visual Uncertainty Improves Multimodal
  Reasoning
Authors: name: Rui Liu, name: Dian Yu, name: Tong Zheng, name: Runpeng Dai, name: Zongxia Li, name: Wenhao Yu, name: Zhenwen Liang, name: Linfeng Song, name: Haitao Mi, name: Pratap Tokekar, name: Dong Yu
Published: 2025-10-01 20:32:08

Reinforcement learning with verifiable rewards (RLVR) improves reasoning in
large language models (LLMs) but struggles with exploration, an issue that
still persists for multimodal LLMs (MLLMs). Current methods treat the visual
input as a fixed, deterministic condition, overlooking a critical source of
ambiguity and struggling to build policies robust to plausible visual
variations. We introduce VOGUE (Visual Uncertainty Guided
Exploration), a novel method that shifts exploration from the output (text)
to the input (visual) space. By treating the image as a stochastic context,
VOGUE quantifies the policy's sensitivity to visual perturbations using the
symmetric KL divergence between a "raw" and "noisy" branch, creating a direct
signal for uncertainty-aware exploration. This signal shapes the learning
objective via an uncertainty-proportional bonus, which, combined with a
token-entropy bonus and an annealed sampling schedule, effectively balances
exploration and exploitation. Implemented within GRPO on two model scales
(Qwen2.5-VL-3B/7B), VOGUE boosts pass@1 accuracy by an average of 2.6% on three
visual math benchmarks and 3.7% on three general-domain reasoning benchmarks,
while simultaneously increasing pass@4 performance and mitigating the
exploration decay commonly observed in RL fine-tuning. Our work shows that
grounding exploration in the inherent uncertainty of visual inputs is an
effective strategy for improving multimodal reasoning.
F2LLM Technical Report: Matching SOTA Embedding Performance with 6
  Million Open-Source Data
Authors: name: Ziyin Zhang, name: Zihan Liao, name: Hang Yu, name: Peng Di, name: Rui Wang
Published: 2025-10-02 17:58:49

We introduce F2LLM - Foundation to Feature Large Language Models, a suite of
state-of-the-art embedding models in three sizes: 0.6B, 1.7B, and 4B. Unlike
previous top-ranking embedding models that require massive contrastive
pretraining, sophisticated training pipelines, and costly synthetic training
data, F2LLM is directly finetuned from foundation models on 6 million
query-document-negative tuples curated from open-source, non-synthetic
datasets, striking a strong balance between training cost, model size, and
embedding performance. On the MTEB English leaderboard, F2LLM-4B ranks 2nd
among models with approximately 4B parameters and 7th overall, while F2LLM-1.7B
ranks 1st among models in the 1B-2B size range. To facilitate future research
in the field, we release the models, training dataset, and code, positioning
F2LLM as a strong, reproducible, and budget-friendly baseline for future works.
Rethinking the shape convention of an MLP
Authors: name: Meng-Hsi Chen, name: Yu-Ang Lee, name: Feng-Ting Liao, name: Da-shan Shiu
Published: 2025-10-02 08:38:15

Multi-layer perceptrons (MLPs) conventionally follow a narrow-wide-narrow
design where skip connections operate at the input/output dimensions while
processing occurs in expanded hidden spaces. We challenge this convention by
proposing wide-narrow-wide (Hourglass) MLP blocks where skip connections
operate at expanded dimensions while residual computation flows through narrow
bottlenecks. This inversion leverages higher-dimensional spaces for incremental
refinement while maintaining computational efficiency through parameter-matched
designs. Implementing Hourglass MLPs requires an initial projection to lift
input signals to expanded dimensions. We propose that this projection can
remain fixed at random initialization throughout training, enabling efficient
training and inference implementations. We evaluate both architectures on
generative tasks over popular image datasets, characterizing
performance-parameter Pareto frontiers through systematic architectural search.
Results show that Hourglass architectures consistently achieve superior Pareto
frontiers compared to conventional designs. As parameter budgets increase,
optimal Hourglass configurations favor deeper networks with wider skip
connections and narrower bottlenecks-a scaling pattern distinct from
conventional MLPs. Our findings suggest reconsidering skip connection placement
in modern architectures, with potential applications extending to Transformers
and other residual networks.
MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment
  Abilities in MLLMs
Authors: name: Jiyao Liu, name: Jinjie Wei, name: Wanying Qu, name: Chenglong Ma, name: Junzhi Ning, name: Yunheng Li, name: Ying Chen, name: Xinzhe Luo, name: Pengcheng Chen, name: Xin Gao, name: Ming Hu, name: Huihui Xu, name: Xin Wang, name: Shujian Gao, name: Dingkang Yang, name: Zhongying Deng, name: Jin Ye, name: Lihao Liu, name: Junjun He, name: Ningsheng Xu
Published: 2025-10-02 05:42:00

Medical Image Quality Assessment (IQA) serves as the first-mile safety gate
for clinical AI, yet existing approaches remain constrained by scalar,
score-based metrics and fail to reflect the descriptive, human-like reasoning
process central to expert evaluation. To address this gap, we introduce
MedQ-Bench, a comprehensive benchmark that establishes a perception-reasoning
paradigm for language-based evaluation of medical image quality with
Multi-modal Large Language Models (MLLMs). MedQ-Bench defines two complementary
tasks: (1) MedQ-Perception, which probes low-level perceptual capability via
human-curated questions on fundamental visual attributes; and (2)
MedQ-Reasoning, encompassing both no-reference and comparison reasoning tasks,
aligning model evaluation with human-like reasoning on image quality. The
benchmark spans five imaging modalities and over forty quality attributes,
totaling 2,600 perceptual queries and 708 reasoning assessments, covering
diverse image sources including authentic clinical acquisitions, images with
simulated degradations via physics-based reconstructions, and AI-generated
images. To evaluate reasoning ability, we propose a multi-dimensional judging
protocol that assesses model outputs along four complementary axes. We further
conduct rigorous human-AI alignment validation by comparing LLM-based judgement
with radiologists. Our evaluation of 14 state-of-the-art MLLMs demonstrates
that models exhibit preliminary but unstable perceptual and reasoning skills,
with insufficient accuracy for reliable clinical use. These findings highlight
the need for targeted optimization of MLLMs in medical IQA. We hope that
MedQ-Bench will catalyze further exploration and unlock the untapped potential
of MLLMs for medical image quality evaluation.
TOUCAN: Synthesizing 1.5M Tool-Agentic Data from Real-World MCP
  Environments
Authors: name: Zhangchen Xu, name: Adriana Meza Soria, name: Shawn Tan, name: Anurag Roy, name: Ashish Sunil Agrawal, name: Radha Poovendran, name: Rameswar Panda
Published: 2025-10-01 17:58:03

Large Language Model (LLM) agents are rapidly emerging as powerful systems
for automating tasks across domains. Yet progress in the open-source community
is constrained by the lack of high quality permissively licensed tool-agentic
training data. Existing datasets are often limited in diversity, realism, and
complexity, particularly regarding multi-tool and multi-turn interactions. To
address this gap, we introduce Toucan, the largest publicly available
tool-agentic dataset to date, containing 1.5 million trajectories synthesized
from nearly 500 real-world Model Context Protocols (MCPs). Unlike prior work,
Toucan leverages authentic MCP environments to generate diverse, realistic, and
challenging tasks with trajectories involving real tool execution. Our pipeline
first produces a broad spectrum of tool-use queries using five distinct models,
applies model-based quality filtering, and then generates agentic trajectories
with three teacher models using two agentic frameworks. Rigorous rule-based and
model-based validation ensures high-quality outputs. We also introduce three
extension mechanisms to further diversify tasks and simulate multi-turn
conversations. Models fine-tuned on Toucan outperform larger closed-source
counterparts on the BFCL V3 benchmark and push the Pareto frontier forward on
MCP-Universe Bench.
VIRTUE: Visual-Interactive Text-Image Universal Embedder
Authors: name: Wei-Yao Wang, name: Kazuya Tateishi, name: Qiyu Wu, name: Shusuke Takahashi, name: Yuki Mitsufuji
Published: 2025-10-01 05:11:54

Multimodal representation learning models have demonstrated successful
operation across complex tasks, and the integration of vision-language models
(VLMs) has further enabled embedding models with instruction-following
capabilities. However, existing embedding models lack visual-interactive
capabilities to specify regions of interest from users (e.g., point, bounding
box, mask), which have been explored in generative models to broaden their
human-interactive applicability. Equipping embedding models with visual
interactions not only would unlock new applications with localized grounding of
user intent, which remains unexplored, but also enable the models to learn
entity-level information within images to complement their global
representations for conventional embedding tasks. In this paper, we propose a
novel Visual-InteRactive Text-Image Universal Embedder (VIRTUE) that extends
the capabilities of the segmentation model and the vision-language model to the
realm of representation learning. In VIRTUE, the segmentation model can process
visual prompts that pinpoint specific regions within an image, thereby enabling
the embedder to handle complex and ambiguous scenarios more precisely. To
evaluate the visual-interaction ability of VIRTUE, we introduce a large-scale
Segmentation-and-Scene Caption Retrieval (SCaR) benchmark comprising 1M samples
that aims to retrieve the text caption by jointly considering the entity with a
specific object and image scene. VIRTUE consistently achieves a
state-of-the-art performance with significant improvements across 36 universal
MMEB (3.1%-8.5%) and five visual-interactive SCaR (15.2%-20.3%) tasks.
DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag
  Editing
Authors: name: Zihan Zhou, name: Shilin Lu, name: Shuli Leng, name: Shaocong Zhang, name: Zhuming Lian, name: Xinlei Yu, name: Adams Wai-Kin Kong
Published: 2025-10-02 17:39:13

Drag-based image editing has long suffered from distortions in the target
region, largely because the priors of earlier base models, Stable Diffusion,
are insufficient to project optimized latents back onto the natural image
manifold. With the shift from UNet-based DDPMs to more scalable DiT with flow
matching (e.g., SD3.5, FLUX), generative priors have become significantly
stronger, enabling advances across diverse editing tasks. However, drag-based
editing has yet to benefit from these stronger priors. This work proposes the
first framework to effectively harness FLUX's rich prior for drag-based
editing, dubbed DragFlow, achieving substantial gains over baselines. We first
show that directly applying point-based drag editing to DiTs performs poorly:
unlike the highly compressed features of UNets, DiT features are insufficiently
structured to provide reliable guidance for point-wise motion supervision. To
overcome this limitation, DragFlow introduces a region-based editing paradigm,
where affine transformations enable richer and more consistent feature
supervision. Additionally, we integrate pretrained open-domain personalization
adapters (e.g., IP-Adapter) to enhance subject consistency, while preserving
background fidelity through gradient mask-based hard constraints. Multimodal
large language models (MLLMs) are further employed to resolve task ambiguities.
For evaluation, we curate a novel Region-based Dragging benchmark (ReD Bench)
featuring region-level dragging instructions. Extensive experiments on
DragBench-DR and ReD Bench show that DragFlow surpasses both point-based and
region-based baselines, setting a new state-of-the-art in drag-based image
editing. Code and datasets will be publicly available upon publication.
Self-Forcing++: Towards Minute-Scale High-Quality Video Generation
Authors: name: Justin Cui, name: Jie Wu, name: Ming Li, name: Tao Yang, name: Xiaojie Li, name: Rui Wang, name: Andrew Bai, name: Yuanhao Ban, name: Cho-Jui Hsieh
Published: 2025-10-02 17:55:42

Diffusion models have revolutionized image and video generation, achieving
unprecedented visual quality. However, their reliance on transformer
architectures incurs prohibitively high computational costs, particularly when
extending generation to long videos. Recent work has explored autoregressive
formulations for long video generation, typically by distilling from
short-horizon bidirectional teachers. Nevertheless, given that teacher models
cannot synthesize long videos, the extrapolation of student models beyond their
training horizon often leads to pronounced quality degradation, arising from
the compounding of errors within the continuous latent space. In this paper, we
propose a simple yet effective approach to mitigate quality degradation in
long-horizon video generation without requiring supervision from long-video
teachers or retraining on long video datasets. Our approach centers on
exploiting the rich knowledge of teacher models to provide guidance for the
student model through sampled segments drawn from self-generated long videos.
Our method maintains temporal consistency while scaling video length by up to
20x beyond teacher's capability, avoiding common issues such as over-exposure
and error-accumulation without recomputing overlapping frames like previous
methods. When scaling up the computation, our method shows the capability of
generating videos up to 4 minutes and 15 seconds, equivalent to 99.9% of the
maximum span supported by our base model's position embedding and more than 50x
longer than that of our baseline model. Experiments on standard benchmarks and
our proposed improved benchmark demonstrate that our approach substantially
outperforms baseline methods in both fidelity and consistency. Our long-horizon
videos demo can be found at https://self-forcing-plus-plus.github.io/
Transformers Discover Molecular Structure Without Graph Priors
Authors: name: Tobias Kreiman, name: Yutong Bai, name: Fadi Atieh, name: Elizabeth Weaver, name: Eric Qu, name: Aditi S. Krishnapriyan
Published: 2025-10-02 17:42:10

Graph Neural Networks (GNNs) are the dominant architecture for molecular
machine learning, particularly for molecular property prediction and machine
learning interatomic potentials (MLIPs). GNNs perform message passing on
predefined graphs often induced by a fixed radius cutoff or k-nearest neighbor
scheme. While this design aligns with the locality present in many molecular
tasks, a hard-coded graph can limit expressivity due to the fixed receptive
field and slows down inference with sparse graph operations. In this work, we
investigate whether pure, unmodified Transformers trained directly on Cartesian
coordinatesx2013without predefined graphs or physical
priorsx2013can approximate molecular energies and forces. As a
starting point for our analysis, we demonstrate how to train a Transformer to
competitive energy and force mean absolute errors under a matched training
compute budget, relative to a state-of-the-art equivariant GNN on the OMol25
dataset. We discover that the Transformer learns physically consistent
patternsx2013such as attention weights that decay inversely with
interatomic distancex2013and flexibly adapts them across different
molecular environments due to the absence of hard-coded biases. The use of a
standard Transformer also unlocks predictable improvements with respect to
scaling training resources, consistent with empirical scaling laws observed in
other domains. Our results demonstrate that many favorable properties of GNNs
can emerge adaptively in Transformers, challenging the necessity of hard-coded
graph inductive biases and pointing toward standardized, scalable architectures
for molecular modeling.
Interactive Training: Feedback-Driven Neural Network Optimization
Authors: name: Wentao Zhang, name: Yang Young Lu, name: Yuntian Deng
Published: 2025-10-02 17:59:00

Traditional neural network training typically follows fixed, predefined
optimization recipes, lacking the flexibility to dynamically respond to
instabilities or emerging training issues. In this paper, we introduce
Interactive Training, an open-source framework that enables real-time,
feedback-driven intervention during neural network training by human experts or
automated AI agents. At its core, Interactive Training uses a control server to
mediate communication between users or agents and the ongoing training process,
allowing users to dynamically adjust optimizer hyperparameters, training data,
and model checkpoints. Through three case studies, we demonstrate that
Interactive Training achieves superior training stability, reduced sensitivity
to initial hyperparameters, and improved adaptability to evolving user needs,
paving the way toward a future training paradigm where AI agents autonomously
monitor training logs, proactively resolve instabilities, and optimize training
dynamics.