Orionnnn/hub-stats huggingface dataset:
Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs
Authors: name: Song Bian, name: Tao Yu, name: Shivaram Venkataraman, name: Youngsuk Park
Published: 2025-10-21 03:08:48

Scaling the number of parameters and the size of training data has proven to
be an effective strategy for improving large language model (LLM) performance.
Yet, as these models grow increasingly powerful and widely deployed, the cost
of inference has become a pressing concern. Despite its importance, the
trade-off between model accuracy and inference efficiency remains
underexplored. In this work, we examine how key architectural factors, hidden
size, the allocation of parameters between MLP and attention (mlp-to-attention
ratio), and grouped-query attention (GQA), influence both inference cost and
accuracy. We introduce a conditional scaling law that augments the Chinchilla
framework with architectural information, along with a search framework for
identifying architectures that are simultaneously inference-efficient and
accurate. To validate our approach, we train more than 200 models spanning 80M
to 3B parameters and 8B to 100B training tokens, and fit the proposed
conditional scaling law. Our results show that the conditional scaling law
reliably predicts optimal architectural choices and that the resulting models
outperform existing open-source baselines. Under the same training budget,
optimized architectures achieve up to 2.1% higher accuracy and 42% greater
inference throughput compared to LLaMA-3.2.
Adamas: Hadamard Sparse Attention for Efficient Long-Context Inference
Authors: name: Siyuan Yan, name: Guo-Qing Jiang, name: Yuchen Zhang, name: Xiaoxing Ma, name: Ran Zhu, name: Chun Cao, name: Jingwei Xu
Published: 2025-10-21 08:44:47

Large language models (LLMs) now support context windows of hundreds of
thousands to millions of tokens, enabling applications such as long-document
summarization, large-scale code synthesis, multi-document question answering
and persistent multi-turn dialogue. However, such extended contexts exacerbate
the quadratic cost of self-attention, leading to severe latency in
autoregressive decoding. Existing sparse attention methods alleviate these
costs but rely on heuristic patterns that struggle to recall critical key-value
(KV) pairs for each query, resulting in accuracy degradation. We introduce
Adamas, a lightweight yet highly accurate sparse attention mechanism designed
for long-context inference. Adamas applies the Hadamard transform,
bucketization and 2-bit compression to produce compact representations, and
leverages Manhattan-distance estimation for efficient top-k selections.
Experiments show that Adamas matches the accuracy of full attention with only a
64-token budget, achieves near-lossless performance at 128, and supports up to
8x higher sparsity than prior state-of-the-art (SOTA) methods while delivering
up to 4.4x self-attention and 1.5x end-to-end speedups on 32K-length sequences.
Remarkably, Adamas attains comparable or even lower perplexity than full
attention, underscoring its effectiveness in maintaining accuracy under
aggressive sparsity.
Long-Context Attention Benchmark: From Kernel Efficiency to Distributed
  Context Parallelism
Authors: name: Tao Bu, name: Qiangang Wang, name: Bowen Zeng, name: Hanwen Sun, name: Yunpeng Huang, name: Chun Cao, name: Jingwei Xu
Published: 2025-10-19 07:07:37

Transformer-based large language models (LLMs) have achieved remarkable
success, yet their standard attention mechanism incurs quadratic computation
and memory costs with respect to sequence length, posing a major bottleneck for
long-context training. Prior work tackles this challenge along two directions:
(1) kernel-level optimizations, which accelerate dense and sparse attention
operators; and (2) module-level strategies, often referred to as distributed
attention or context parallel training, which scale attention across multiple
devices. However, systematic evaluation still remains limited: operator-level
comparisons are often incomplete, while context parallel strategies are
typically framework-specific, with unclear performance analysis across
contexts. To address these gaps, we propose a unified benchmark that integrates
representative attention kernels and context parallel mechanisms with a modular
and extensible interface for evaluation. The benchmark evaluates methods along
two critical dimensions: (1) attention mask patterns, which strongly affect
efficiency, scalability, and usability, and (2) sequence length and distributed
scale, which determine performance under extreme long-context training. Through
comprehensive experiments on the cluster of up to 96 GPUs, our benchmark
enables reproducible comparisons, highlights method-specific trade-offs, and
provides practical guidance for designing and deploying attention mechanisms in
long-context LLM training.
CiteGuard: Faithful Citation Attribution for LLMs via
  Retrieval-Augmented Validation
Authors: name: Yee Man Choi, name: Xuehang Guo, name: Yi R., name: Fung, name: Qingyun Wang
Published: 2025-10-15 00:32:26

Large Language Models (LLMs) have emerged as promising assistants for
scientific writing. However, there have been concerns regarding the quality and
reliability of the generated text, one of which is the citation accuracy and
faithfulness. While most recent work relies on methods such as LLM-as-a-Judge,
the reliability of LLM-as-a-Judge alone is also in doubt. In this work, we
reframe citation evaluation as a problem of citation attribution alignment,
which is assessing whether LLM-generated citations match those a human author
would include for the same text. We propose CiteGuard, a retrieval-aware agent
framework designed to provide more faithful grounding for citation validation.
CiteGuard improves the prior baseline by 12.3%, and achieves up to 65.4%
accuracy on the CiteME benchmark, on par with human-level performance (69.7%).
It also enables the identification of alternative but valid citations.
Communication to Completion: Modeling Collaborative Workflows with
  Intelligent Multi-Agent Communication
Authors: name: Yiming Lu, name: Xun Wang, name: Simin Ma, name: Shujian Liu, name: Sathish Reddy Indurthi, name: Song Wang, name: Haoyun Deng, name: Fei Liu, name: Kaiqiang Song
Published: 2025-10-22 19:48:17

Teamwork in workspace for complex tasks requires diverse communication
strategies, but current multi-agent LLM systems lack systematic frameworks for
task oriented communication. We introduce Communication to Completion (C2C), a
scalable framework that addresses this gap through two key innovations: (1) the
Alignment Factor (AF), a novel metric quantifying agent task alignment that
directly impacts work efficiency, and (2) a Sequential Action Framework that
integrates stepwise execution with intelligent communication decisions. C2C
enables agents to make cost aware communication choices, dynamically improving
task understanding through targeted interactions. We evaluated C2C on realistic
coding workflows across three complexity tiers and team sizes from 5 to 17
agents, comparing against no communication and fixed steps baselines. The
results show that C2C reduces the task completion time by about 40% with
acceptable communication costs. The framework completes all tasks successfully
in standard configurations and maintains effectiveness at scale. C2C
establishes both a theoretical foundation for measuring communication
effectiveness in multi-agent systems and a practical framework for complex
collaborative tasks.
ComProScanner: A multi-agent based framework for composition-property
  structured data extraction from scientific literature
Authors: name: Aritra Roy, name: Enrico Grisan, name: John Buckeridge, name: Chiara Gattinoni
Published: 2025-10-23 09:01:44

Since the advent of various pre-trained large language models, extracting
structured knowledge from scientific text has experienced a revolutionary
change compared with traditional machine learning or natural language
processing techniques. Despite these advances, accessible automated tools that
allow users to construct, validate, and visualise datasets from scientific
literature extraction remain scarce. We therefore developed ComProScanner, an
autonomous multi-agent platform that facilitates the extraction, validation,
classification, and visualisation of machine-readable chemical compositions and
properties, integrated with synthesis data from journal articles for
comprehensive database creation. We evaluated our framework using 100 journal
articles against 10 different LLMs, including both open-source and proprietary
models, to extract highly complex compositions associated with ceramic
piezoelectric materials and corresponding piezoelectric strain coefficients
(d33), motivated by the lack of a large dataset for such materials.
DeepSeek-V3-0324 outperformed all models with a significant overall accuracy of
0.82. This framework provides a simple, user-friendly, readily-usable package
for extracting highly complex experimental data buried in the literature to
build machine learning or deep learning datasets.
Diff-XYZ: A Benchmark for Evaluating Diff Understanding
Authors: name: Evgeniy Glukhov, name: Michele Conti, name: Egor Bogomolov, name: Yaroslav Golubev, name: Alexander Bezzubov
Published: 2025-10-14 13:23:01

Reliable handling of code diffs is central to agents that edit and refactor
repositories at scale. We introduce Diff-XYZ, a compact benchmark for code-diff
understanding with three supervised tasks: apply (old code + diff
rightarrow new code), anti-apply (new code - diff rightarrow old code),
and diff generation (new code - old code rightarrow diff). Instances in
the benchmark are triples langle old code, new code,
diff rangle drawn from real commits in CommitPackFT, paired with
automatic metrics and a clear evaluation protocol. We use the benchmark to do a
focused empirical study of the unified diff format and run a cross-format
comparison of different diff representations. Our findings reveal that
different formats should be used depending on the use case and model size. For
example, representing diffs in search-replace format is good for larger models
in the diff generation scenario, yet not suited well for diff analysis and
smaller models. The Diff-XYZ benchmark is a reusable foundation for assessing
and improving diff handling in LLMs that can aid future development of diff
formats and models editing code. The dataset is published on HuggingFace Hub:
https://huggingface.co/datasets/JetBrains-Research/diff-xyz.
SAKE: Towards Editing Auditory Attribute Knowledge of Large
  Audio-Language Models
Authors: name: Chih-Kai Yang, name: Yen-Ting Piao, name: Tzu-Wen Hsu, name: Szu-Wei Fu, name: Zhehuai Chen, name: Ke-Han Lu, name: Sung-Feng Huang, name: Chao-Han Huck Yang, name: Yu-Chiang Frank Wang, name: Yun-Nung Chen, name: Hung-yi Lee
Published: 2025-10-19 16:22:09

Knowledge editing offers an efficient way to update model knowledge without
full retraining, but prior work has concentrated almost exclusively on textual
or visual modalities. We introduce SAKE, the first benchmark specifically
designed for editing auditory attribute knowledge in Large Audio-Language
Models (LALMs). Unlike factual updates, SAKE targets several abstract auditory
attributes, capturing knowledge types that go beyond conventional textual and
visual domains. We benchmark seven editing methods on two LALMs along four
dimensions: reliability, generality, audio/text locality, and portability.
Results highlight challenges such as preserving intra-attribute knowledge
unrelated to the edit, generalizing edits to multimodal reasoning, and
maintaining edits under sequential updates. SAKE provides a principled
framework to study how knowledge editing extends to the auditory modalities,
opening new directions for maintaining and adapting LALMs in more diverse
real-world scenarios.
Investigating Safety Vulnerabilities of Large Audio-Language Models
  Under Speaker Emotional Variations
Authors: name: Bo-Han Feng, name: Chien-Feng Liu, name: Yu-Hsuan Li Liang, name: Chih-Kai Yang, name: Szu-Wei Fu, name: Zhehuai Chen, name: Ke-Han Lu, name: Sung-Feng Huang, name: Chao-Han Huck Yang, name: Yu-Chiang Frank Wang, name: Yun-Nung Chen, name: Hung-yi Lee
Published: 2025-10-19 15:41:25

Large audio-language models (LALMs) extend text-based LLMs with auditory
understanding, offering new opportunities for multimodal applications. While
their perception, reasoning, and task performance have been widely studied,
their safety alignment under paralinguistic variation remains underexplored.
This work systematically investigates the role of speaker emotion. We construct
a dataset of malicious speech instructions expressed across multiple emotions
and intensities, and evaluate several state-of-the-art LALMs. Our results
reveal substantial safety inconsistencies: different emotions elicit varying
levels of unsafe responses, and the effect of intensity is non-monotonic, with
medium expressions often posing the greatest risk. These findings highlight an
overlooked vulnerability in LALMs and call for alignment strategies explicitly
designed to ensure robustness under emotional variation, a prerequisite for
trustworthy deployment in real-world settings.
Thought Communication in Multiagent Collaboration
Authors: name: Yujia Zheng, name: Zhuokai Zhao, name: Zijian Li, name: Yaqi Xie, name: Mingze Gao, name: Lizhu Zhang, name: Kun Zhang
Published: 2025-10-23 16:48:02

Natural language has long enabled human cooperation, but its lossy,
ambiguous, and indirect nature limits the potential of collective intelligence.
While machines are not subject to these constraints, most LLM-based multi-agent
systems still rely solely on natural language, exchanging tokens or their
embeddings. To go beyond language, we introduce a new paradigm, thought
communication, which enables agents to interact directly mind-to-mind, akin to
telepathy. To uncover these latent thoughts in a principled way, we formalize
the process as a general latent variable model, where agent states are
generated by an unknown function of underlying thoughts. We prove that, in a
nonparametric setting without auxiliary information, both shared and private
latent thoughts between any pair of agents can be identified. Moreover, the
global structure of thought sharing, including which agents share which
thoughts and how these relationships are structured, can also be recovered with
theoretical guarantees. Guided by the established theory, we develop a
framework that extracts latent thoughts from all agents prior to communication
and assigns each agent the relevant thoughts, along with their sharing
patterns. This paradigm naturally extends beyond LLMs to all modalities, as
most observational data arise from hidden generative processes. Experiments on
both synthetic and real-world benchmarks validate the theory and demonstrate
the collaborative advantages of thought communication. We hope this work
illuminates the potential of leveraging the hidden world, as many challenges
remain unsolvable through surface-level observation alone, regardless of
compute or data scale.
Seed3D 1.0: From Images to High-Fidelity Simulation-Ready 3D Assets
Authors: name: Jiashi Feng, name: Xiu Li, name: Jing Lin, name: Jiahang Liu, name: Gaohong Liu, name: Weiqiang Lou, name: Su Ma, name: Guang Shi, name: Qinlong Wang, name: Jun Wang, name: Zhongcong Xu, name: Xuanyu Yi, name: Zihao Yu, name: Jianfeng Zhang, name: Yifan Zhu, name: Rui Chen, name: Jinxin Chi, name: Zixian Du, name: Li Han, name: Lixin Huang, name: Kaihua Jiang, name: Yuhan Li, name: Guan Luo, name: Shuguang Wang, name: Qianyi Wu, name: Fan Yang, name: Junyang Zhang, name: Xuanmeng Zhang
Published: 2025-10-22 18:16:32

Developing embodied AI agents requires scalable training environments that
balance content diversity with physics accuracy. World simulators provide such
environments but face distinct limitations: video-based methods generate
diverse content but lack real-time physics feedback for interactive learning,
while physics-based engines provide accurate dynamics but face scalability
limitations from costly manual asset creation. We present Seed3D 1.0, a
foundation model that generates simulation-ready 3D assets from single images,
addressing the scalability challenge while maintaining physics rigor. Unlike
existing 3D generation models, our system produces assets with accurate
geometry, well-aligned textures, and realistic physically-based materials.
These assets can be directly integrated into physics engines with minimal
configuration, enabling deployment in robotic manipulation and simulation
training. Beyond individual objects, the system scales to complete scene
generation through assembling objects into coherent environments. By enabling
scalable simulation-ready content creation, Seed3D 1.0 provides a foundation
for advancing physics-based world simulators. Seed3D 1.0 is now available on
https://console.volcengine.com/ark/region:ark+cn-beijing/experience/vision?modelId=doubao-seed3d-1-0-250928&tab=Gen3D
From Masks to Worlds: A Hitchhiker's Guide to World Models
Authors: name: Jinbin Bai, name: Yu Lei, name: Hecong Wu, name: Yuchen Zhu, name: Shufan Li, name: Yi Xin, name: Xiangtai Li, name: Molei Tao, name: Aditya Grover, name: Ming-Hsuan Yang
Published: 2025-10-23 15:46:44

This is not a typical survey of world models; it is a guide for those who
want to build worlds. We do not aim to catalog every paper that has ever
mentioned a ``world model". Instead, we follow one clear road: from early
masked models that unified representation learning across modalities, to
unified architectures that share a single paradigm, then to interactive
generative models that close the action-perception loop, and finally to
memory-augmented systems that sustain consistent worlds over time. We bypass
loosely related branches to focus on the core: the generative heart, the
interactive loop, and the memory system. We show that this is the most
promising path towards true world models.
Search Self-play: Pushing the Frontier of Agent Capability without
  Supervision
Authors: name: Hongliang Lu, name: Yuhang Wen, name: Pengyu Cheng, name: Ruijin Ding, name: Haotian Xu, name: Jiaqi Guo, name: Chutian Wang, name: Haonan Chen, name: Xiaoxi Jiang, name: Guanjun Jiang
Published: 2025-10-21 17:19:35

Reinforcement learning with verifiable rewards (RLVR) has become the
mainstream technique for training LLM agents. However, RLVR highly depends on
well-crafted task queries and corresponding ground-truth answers to provide
accurate rewards, which requires massive human efforts and hinders the RL
scaling processes, especially under agentic scenarios. Although a few recent
works explore task synthesis methods, the difficulty of generated agentic tasks
can hardly be controlled to provide effective RL training advantages. To
achieve agentic RLVR with higher scalability, we explore self-play training for
deep search agents, in which the learning LLM utilizes multi-turn search engine
calling and acts simultaneously as both a task proposer and a problem solver.
The task proposer aims to generate deep search queries with well-defined
ground-truth answers and increasing task difficulty. The problem solver tries
to handle the generated search queries and output the correct answer
predictions. To ensure that each generated search query has accurate ground
truth, we collect all the searching results from the proposer's trajectory as
external knowledge, then conduct retrieval-augmentation generation (RAG) to
test whether the proposed query can be correctly answered with all necessary
search documents provided. In this search self-play (SSP) game, the proposer
and the solver co-evolve their agent capabilities through both competition and
cooperation. With substantial experimental results, we find that SSP can
significantly improve search agents' performance uniformly on various
benchmarks without any supervision under both from-scratch and continuous RL
training setups. The code is at https://github.com/Alibaba-Quark/SSP.
Emergence of Linear Truth Encodings in Language Models
Authors: name: Shauli Ravfogel, name: Gilad Yehudai, name: Tal Linzen, name: Joan Bruna, name: Alberto Bietti
Published: 2025-10-17 16:30:07

Recent probing studies reveal that large language models exhibit linear
subspaces that separate true from false statements, yet the mechanism behind
their emergence is unclear. We introduce a transparent, one-layer transformer
toy model that reproduces such truth subspaces end-to-end and exposes one
concrete route by which they can arise. We study one simple setting in which
truth encoding can emerge: a data distribution where factual statements
co-occur with other factual statements (and vice-versa), encouraging the model
to learn this distinction in order to lower the LM loss on future tokens. We
corroborate this pattern with experiments in pretrained language models.
Finally, in the toy setting we observe a two-phase learning dynamic: networks
first memorize individual factual associations in a few steps, then -- over a
longer horizon -- learn to linearly separate true from false, which in turn
lowers language-modeling loss. Together, these results provide both a
mechanistic demonstration and an empirical motivation for how and why linear
truth representations can emerge in language models.
The Massive Legal Embedding Benchmark (MLEB)
Authors: name: Umar Butler, name: Abdur-Rahman Butler, name: Adrian Lucas Malec
Published: 2025-10-22 08:38:44

We present the Massive Legal Embedding Benchmark (MLEB), the largest, most
diverse, and most comprehensive open-source benchmark for legal information
retrieval to date. MLEB consists of ten expert-annotated datasets spanning
multiple jurisdictions (the US, UK, EU, Australia, Ireland, and Singapore),
document types (cases, legislation, regulatory guidance, contracts, and
literature), and task types (search, zero-shot classification, and question
answering). Seven of the datasets in MLEB were newly constructed in order to
fill domain and jurisdictional gaps in the open-source legal information
retrieval landscape. We document our methodology in building MLEB and creating
the new constituent datasets, and release our code, results, and data openly to
assist with reproducible evaluations.
DyPE: Dynamic Position Extrapolation for Ultra High Resolution Diffusion
Authors: name: Noam Issachar, name: Guy Yariv, name: Sagie Benaim, name: Yossi Adi, name: Dani Lischinski, name: Raanan Fattal
Published: 2025-10-23 17:42:14

Diffusion Transformer models can generate images with remarkable fidelity and
detail, yet training them at ultra-high resolutions remains extremely costly
due to the self-attention mechanism's quadratic scaling with the number of
image tokens. In this paper, we introduce Dynamic Position Extrapolation
(DyPE), a novel, training-free method that enables pre-trained diffusion
transformers to synthesize images at resolutions far beyond their training
data, with no additional sampling cost. DyPE takes advantage of the spectral
progression inherent to the diffusion process, where low-frequency structures
converge early, while high-frequencies take more steps to resolve.
Specifically, DyPE dynamically adjusts the model's positional encoding at each
diffusion step, matching their frequency spectrum with the current stage of the
generative process. This approach allows us to generate images at resolutions
that exceed the training resolution dramatically, e.g., 16 million pixels using
FLUX. On multiple benchmarks, DyPE consistently improves performance and
achieves state-of-the-art fidelity in ultra-high-resolution image generation,
with gains becoming even more pronounced at higher resolutions. Project page is
available at https://noamissachar.github.io/DyPE/.