Orionnnn/hub-stats huggingface dataset:
Continuous Autoregressive Language Models
Authors: name: Chenze Shao, name: Darren Li, name: Fandong Meng, name: Jie Zhou
Published: 2025-10-31 17:58:11

The efficiency of large language models (LLMs) is fundamentally limited by
their sequential, token-by-token generation process. We argue that overcoming
this bottleneck requires a new design axis for LLM scaling: increasing the
semantic bandwidth of each generative step. To this end, we introduce
Continuous Autoregressive Language Models (CALM), a paradigm shift from
discrete next-token prediction to continuous next-vector prediction. CALM uses
a high-fidelity autoencoder to compress a chunk of K tokens into a single
continuous vector, from which the original tokens can be reconstructed with
over 99.9\% accuracy. This allows us to model language as a sequence of
continuous vectors instead of discrete tokens, which reduces the number of
generative steps by a factor of K. The paradigm shift necessitates a new
modeling toolkit; therefore, we develop a comprehensive likelihood-free
framework that enables robust training, evaluation, and controllable sampling
in the continuous domain. Experiments show that CALM significantly improves the
performance-compute trade-off, achieving the performance of strong discrete
baselines at a significantly lower computational cost. More importantly, these
findings establish next-vector prediction as a powerful and scalable pathway
towards ultra-efficient language models. Code:
https://github.com/shaochenze/calm. Project:
https://shaochenze.github.io/blog/2025/CALM.
The Denario project: Deep knowledge AI agents for scientific discovery
Authors: name: Francisco Villaescusa-Navarro, name: Boris Bolliet, name: Pablo Villanueva-Domingo, name: Adrian E. Bayer, name: Aidan Acquah, name: Chetana Amancharla, name: Almog Barzilay-Siegal, name: Pablo Bermejo, name: Camille Bilodeau, name: Pablo Cárdenas Ramírez, name: Miles Cranmer, name: Urbano L. França, name: ChangHoon Hahn, name: Yan-Fei Jiang, name: Raul Jimenez, name: Jun-Young Lee, name: Antonio Lerario, name: Osman Mamun, name: Thomas Meier, name: Anupam A. Ojha, name: Pavlos Protopapas, name: Shimanto Roy, name: David N. Spergel, name: Pedro Tarancón-Álvarez, name: Ujjwal Tiwari, name: Matteo Viel, name: Digvijay Wadekar, name: Chi Wang, name: Bonny Y. Wang, name: Licong Xu, name: Yossi Yovel, name: Shuwen Yue, name: Wen-Han Zhou, name: Qiyao Zhu, name: Jiajun Zou, name: Íñigo Zubeldia
Published: 2025-10-30 18:00:12

We present Denario, an AI multi-agent system designed to serve as a
scientific research assistant. Denario can perform many different tasks, such
as generating ideas, checking the literature, developing research plans,
writing and executing code, making plots, and drafting and reviewing a
scientific paper. The system has a modular architecture, allowing it to handle
specific tasks, such as generating an idea, or carrying out end-to-end
scientific analysis using Cmbagent as a deep-research backend. In this work, we
describe in detail Denario and its modules, and illustrate its capabilities by
presenting multiple AI-generated papers generated by it in many different
scientific disciplines such as astrophysics, biology, biophysics, biomedical
informatics, chemistry, material science, mathematical physics, medicine,
neuroscience and planetary science. Denario also excels at combining ideas from
different disciplines, and we illustrate this by showing a paper that applies
methods from quantum physics and machine learning to astrophysical data. We
report the evaluations performed on these papers by domain experts, who
provided both numerical scores and review-like feedback. We then highlight the
strengths, weaknesses, and limitations of the current system. Finally, we
discuss the ethical implications of AI-driven research and reflect on how such
technology relates to the philosophy of science. We publicly release the code
at https://github.com/AstroPilot-AI/Denario. A Denario demo can also be run
directly on the web at https://huggingface.co/spaces/astropilot-ai/Denario, and
the full app will be deployed on the cloud.
Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action
  Model
Authors: name: John Won, name: Kyungmin Lee, name: Huiwon Jang, name: Dongyoung Kim, name: Jinwoo Shin
Published: 2025-10-31 16:32:12

Recently, augmenting Vision-Language-Action models (VLAs) with world modeling
has shown promise in improving robotic policy learning. However, it remains
challenging to jointly predict next-state observations and action sequences
because of the inherent difference between the two modalities. To address this,
we propose DUal-STream diffusion (DUST), a world-model augmented VLA framework
that handles the modality conflict and enhances the performance of VLAs across
diverse tasks. Specifically, we propose a multimodal diffusion transformer
architecture that explicitly maintains separate modality streams while still
enabling cross-modal knowledge sharing. In addition, we introduce independent
noise perturbations for each modality and a decoupled flow-matching loss. This
design enables the model to learn the joint distribution in a bidirectional
manner while avoiding the need for a unified latent space. Based on the
decoupling of modalities during training, we also introduce a joint sampling
method that supports test-time scaling, where action and vision tokens evolve
asynchronously at different rates. Through experiments on simulated benchmarks
such as RoboCasa and GR-1, DUST achieves up to 6% gains over baseline methods,
while our test-time scaling approach provides an additional 2-5% boost. On
real-world tasks with the Franka Research 3, DUST improves success rates by
13%, confirming its effectiveness beyond simulation. Furthermore, pre-training
on action-free videos from BridgeV2 yields significant transfer gains on
RoboCasa, underscoring DUST's potential for large-scale VLA pretraining.
POWSM: A Phonetic Open Whisper-Style Speech Foundation Model
Authors: name: Chin-Jou Li, name: Kalvin Chang, name: Shikhar Bharadwaj, name: Eunjung Yeo, name: Kwanghee Choi, name: Jian Zhu, name: David Mortensen, name: Shinji Watanabe
Published: 2025-10-28 21:43:45

Recent advances in spoken language processing have led to substantial
progress in phonetic tasks such as automatic speech recognition (ASR), phone
recognition (PR), grapheme-to-phoneme conversion (G2P), and phoneme-to-grapheme
conversion (P2G). Despite their conceptual similarity, these tasks have largely
been studied in isolation, each relying on task-specific architectures and
datasets. In this paper, we introduce POWSM (Phonetic Open Whisper-style Speech
Model), the first unified framework capable of jointly performing multiple
phone-related tasks. POWSM enables seamless conversion between audio, text
(graphemes), and phones, opening up new possibilities for universal and
low-resource speech processing. Our model outperforms or matches specialized PR
models of similar size (Wav2Vec2Phoneme and ZIPA) while jointly supporting G2P,
P2G, and ASR. Our training data, code and models are released to foster open
science.
L^2M^3OF: A Large Language Multimodal Model for Metal-Organic Frameworks
Authors: name: Jiyu Cui, name: Fang Wu, name: Haokai Zhao, name: Minggao Feng, name: Xenophon Evangelopoulos, name: Andrew I. Cooper, name: Yejin Choi
Published: 2025-10-23 20:12:46

Large language models have demonstrated remarkable reasoning capabilities
across diverse natural language tasks. However, comparable breakthroughs in
scientific discovery are more limited, because understanding complex physical
phenomena demands multifaceted representations far beyond language alone. A
compelling example is the design of functional materials such as MOFs-critical
for a range of impactful applications like carbon capture and hydrogen storage.
Navigating their vast and intricate design space in language-based
representations interpretable by LLMs is challenging due to the numerous
possible three-dimensional atomic arrangements and strict reticular rules of
coordination geometry and topology. Despite promising early results in
LLM-assisted discovery for simpler materials systems, MOF design remains
heavily reliant on tacit human expertise rarely codified in textual information
alone. To overcome this barrier, we introduce L2M3OF, the first multimodal LLM
for MOFs. L2M3OF integrates crystal representation learning with language
understanding to process structural, textual, and knowledge modalities jointly.
L2M3OF employs a pre-trained crystal encoder with a lightweight projection
layer to compress structural information into a token space, enabling efficient
alignment with language instructions. To facilitate training and evaluation, we
curate a structure-property-knowledge database of crystalline materials and
benchmark L2M3OF against state-of-the-art closed-source LLMs such as GPT-5,
Gemini-2.5-Pro and DeepSeek-R1. Experiments show that L2M3OF outperforms
leading text-based closed-source LLMs in property prediction and knowledge
generation tasks, despite using far fewer parameters. These results highlight
the importance of multimodal approaches for porous material understanding and
establish L2M3OF as a foundation for next-generation AI systems in materials
discovery.
Performance Trade-offs of Optimizing Small Language Models for
  E-Commerce
Authors: name: Josip Tomo Licardo, name: Nikola Tankovic
Published: 2025-10-24 18:49:28

Large Language Models (LLMs) offer state-of-the-art performance in natural
language understanding and generation tasks. However, the deployment of leading
commercial models for specialized tasks, such as e-commerce, is often hindered
by high computational costs, latency, and operational expenses. This paper
investigates the viability of smaller, open-weight models as a
resource-efficient alternative. We present a methodology for optimizing a
one-billion-parameter Llama 3.2 model for multilingual e-commerce intent
recognition. The model was fine-tuned using Quantized Low-Rank Adaptation
(QLoRA) on a synthetically generated dataset designed to mimic real-world user
queries. Subsequently, we applied post-training quantization techniques,
creating GPU-optimized (GPTQ) and CPU-optimized (GGUF) versions. Our results
demonstrate that the specialized 1B model achieves 99% accuracy, matching the
performance of the significantly larger GPT-4.1 model. A detailed performance
analysis revealed critical, hardware-dependent trade-offs: while 4-bit GPTQ
reduced VRAM usage by 41%, it paradoxically slowed inference by 82% on an older
GPU architecture (NVIDIA T4) due to dequantization overhead. Conversely, GGUF
formats on a CPU achieved a speedup of up to 18x in inference throughput and a
reduction of over 90% in RAM consumption compared to the FP16 baseline. We
conclude that small, properly optimized open-weight models are not just a
viable but a more suitable alternative for domain-specific applications,
offering state-of-the-art accuracy at a fraction of the computational cost.
PORTool: Tool-Use LLM Training with Rewarded Tree
Authors: name: Feijie Wu, name: Weiwu Zhu, name: Yuxiang Zhang, name: Soumya Chatterjee, name: Jiarong Zhu, name: Fan Mo, name: Rodin Luo, name: Jing Gao
Published: 2025-10-29 23:28:53

Current tool-use large language models (LLMs) are trained on static datasets,
enabling them to interact with external tools and perform multi-step,
tool-integrated reasoning, which produces tool-call trajectories. However,
these models imitate how a query is resolved in a generic tool-call routine,
thereby failing to explore possible solutions and demonstrating limited
performance in an evolved, dynamic tool-call environment. In this work, we
propose PORTool, a reinforcement learning (RL) method that encourages a
tool-use LLM to explore various trajectories yielding the correct answer.
Specifically, this method starts with generating multiple rollouts for a given
query, and some of them share the first few tool-call steps, thereby forming a
tree-like structure. Next, we assign rewards to each step, based on its ability
to produce a correct answer and make successful tool calls. A shared step
across different trajectories receives the same reward, while different steps
under the same fork receive different rewards. Finally, these step-wise rewards
are used to calculate fork-relative advantages, blended with
trajectory-relative advantages, to train the LLM for tool use. The experiments
utilize 17 tools to address user queries, covering both time-sensitive and
time-invariant topics. We conduct ablation studies to systematically justify
the necessity and the design robustness of step-wise rewards. Furthermore, we
compare the proposed PORTool with other training approaches and demonstrate
significant improvements in final accuracy and the number of tool-call steps.
CityRiSE: Reasoning Urban Socio-Economic Status in Vision-Language
  Models via Reinforcement Learning
Authors: name: Tianhui Liu, name: Hetian Pang, name: Xin Zhang, name: Jie Feng, name: Yong Li, name: Pan Hui
Published: 2025-10-25 12:56:46

Harnessing publicly available, large-scale web data, such as street view and
satellite imagery, urban socio-economic sensing is of paramount importance for
achieving global sustainable development goals. With the emergence of Large
Vision-Language Models (LVLMs), new opportunities have arisen to solve this
task by treating it as a multi-modal perception and understanding problem.
However, recent studies reveal that LVLMs still struggle with accurate and
interpretable socio-economic predictions from visual data. To address these
limitations and maximize the potential of LVLMs, we introduce
CityRiSE, a novel framework for Reasoning urban
Socio-Economic status in LVLMs through pure reinforcement
learning (RL). With carefully curated multi-modal data and verifiable reward
design, our approach guides the LVLM to focus on semantically meaningful visual
cues, enabling structured and goal-oriented reasoning for generalist
socio-economic status prediction. Experiments demonstrate that CityRiSE with
emergent reasoning process significantly outperforms existing baselines,
improving both prediction accuracy and generalization across diverse urban
contexts, particularly for prediction on unseen cities and unseen indicators.
This work highlights the promise of combining RL and LVLMs for interpretable
and generalist urban socio-economic sensing.
The End of Manual Decoding: Towards Truly End-to-End Language Models
Authors: name: Zhichao Wang, name: Dongyang Ma, name: Xinting Huang, name: Deng Cai, name: Tian Lan, name: Jiahao Xu, name: Haitao Mi, name: Xiaoying Tang, name: Yan Wang
Published: 2025-10-30 17:01:43

The "end-to-end" label for LLMs is a misnomer. In practice, they depend on a
non-differentiable decoding process that requires laborious, hand-tuning of
hyperparameters like temperature and top-p. This paper introduces AutoDeco, a
novel architecture that enables truly "end-to-end" generation by learning to
control its own decoding strategy. We augment the standard transformer with
lightweight heads that, at each step, dynamically predict context-specific
temperature and top-p values alongside the next-token logits. This approach
transforms decoding into a parametric, token-level process, allowing the model
to self-regulate its sampling strategy within a single forward pass.
  Through extensive experiments on eight benchmarks, we demonstrate that
AutoDeco not only significantly outperforms default decoding strategies but
also achieves performance comparable to an oracle-tuned baseline derived from
"hacking the test set"-a practical upper bound for any static method.
Crucially, we uncover an emergent capability for instruction-based decoding
control: the model learns to interpret natural language commands (e.g.,
"generate with low randomness") and adjusts its predicted temperature and top-p
on a token-by-token basis, opening a new paradigm for steerable and interactive
LLM decoding.
MedVLSynther: Synthesizing High-Quality Visual Question Answering from
  Medical Documents with Generator-Verifier LMMs
Authors: name: Xiaoke Huang, name: Ningsen Wang, name: Hui Liu, name: Xianfeng Tang, name: Yuyin Zhou
Published: 2025-10-29 18:10:44

Large Multimodal Models (LMMs) are increasingly capable of answering medical
questions that require joint reasoning over images and text, yet training
general medical VQA systems is impeded by the lack of large, openly usable,
high-quality corpora. We present MedVLSynther, a rubric-guided
generator-verifier framework that synthesizes high-quality multiple-choice VQA
items directly from open biomedical literature by conditioning on figures,
captions, and in-text references. The generator produces self-contained stems
and parallel, mutually exclusive options under a machine-checkable JSON schema;
a multi-stage verifier enforces essential gates (self-containment, single
correct answer, clinical validity, image-text consistency), awards fine-grained
positive points, and penalizes common failure modes before acceptance. Applying
this pipeline to PubMed Central yields MedSynVQA: 13,087 audited questions over
14,803 images spanning 13 imaging modalities and 28 anatomical regions.
Training open-weight LMMs with reinforcement learning using verifiable rewards
improves accuracy across six medical VQA benchmarks, achieving averages of
55.85 (3B) and 58.15 (7B), with up to 77.57 on VQA-RAD and 67.76 on PathVQA,
outperforming strong medical LMMs. A Ablations verify that both generation and
verification are necessary and that more verified data consistently helps, and
a targeted contamination analysis detects no leakage from evaluation suites. By
operating entirely on open literature and open-weight models, MedVLSynther
offers an auditable, reproducible, and privacy-preserving path to scalable
medical VQA training data.
CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction
  Tuning for BabyLMs
Authors: name: Luca Capone, name: Alessandro Bondielli, name: Alessandro Lenci
Published: 2025-10-29 10:36:39

This work investigates whether small-scale LMs can benefit from instruction
tuning. We compare conversational and question-answering instruction tuning
datasets, applied either in a merged or sequential curriculum, using
decoder-only models with 100M and 140M parameters. Evaluation spans both
fine-tuning (SuperGLUE) and zero-shot (BLiMP, EWoK, WUGs, entity tracking, and
psycholinguistic correlation) settings. Results show that instruction tuning
yields small but consistent gains in fine-tuning scenarios, with sequential
curricula outperforming merged data; however, improvements do not consistently
transfer to zero-shot tasks, suggesting a trade-off between interaction-focused
adaptation and broad linguistic generalization. These results highlight both
the potential and the constraints of adapting human-inspired learning
strategies to low-resource LMs, and point toward hybrid, curriculum-based
approaches for enhancing generalization under ecological training limits.
Surfer 2: The Next Generation of Cross-Platform Computer Use Agents
Authors: name: Mathieu Andreux, name: Märt Bakler, name: Yanael Barbier, name: Hamza Benchekroun, name: Emilien Biré, name: Antoine Bonnet, name: Riaz Bordie, name: Nathan Bout, name: Matthias Brunel, name: Aleix Cambray, name: Pierre-Louis Cedoz, name: Antoine Chassang, name: Gautier Cloix, name: Ethan Connelly, name: Alexandra Constantinou, name: Ramzi De Coster, name: Hubert de la Jonquiere, name: Aurélien Delfosse, name: Maxime Delpit, name: Alexis Deprez, name: Augustin Derupti, name: Mathieu Diaz, name: Shannon D'Souza, name: Julie Dujardin, name: Abai Edmund, name: Michael Eickenberg, name: Armand Fatalot, name: Wissem Felissi, name: Isaac Herring, name: Xavier Koegler, name: Erwan Le Jumeau de Kergaradec, name: Aurélien Lac, name: Maxime Langevin, name: Corentin Lauverjat, name: Antonio Loison, name: Avshalom Manevich, name: Axel Moyal, name: Axel Nguyen Kerbel, name: Marinela Parovic, name: Julien Revelle, name: Guillaume Richard, name: Mats Richter, name: Ronan Riochet, name: María Santos, name: Romain Savidan, name: Laurent Sifre, name: Maxime Theillard, name: Marc Thibault, name: Ivan Valentini, name: Tony Wu, name: Laura Yie, name: Kai Yuan, name: Jevgenij Zubovskij
Published: 2025-10-22 18:21:52

Building agents that generalize across web, desktop, and mobile environments
remains an open challenge, as prior systems rely on environment-specific
interfaces that limit cross-platform deployment. We introduce Surfer 2, a
unified architecture operating purely from visual observations that achieves
state-of-the-art performance across all three environments. Surfer 2 integrates
hierarchical context management, decoupled planning and execution, and
self-verification with adaptive recovery, enabling reliable operation over long
task horizons. Our system achieves 97.1% accuracy on WebVoyager, 69.6% on
WebArena, 60.1% on OSWorld, and 87.1% on AndroidWorld, outperforming all prior
systems without task-specific fine-tuning. With multiple attempts, Surfer 2
exceeds human performance on all benchmarks. These results demonstrate that
systematic orchestration amplifies foundation model capabilities and enables
general-purpose computer control through visual interaction alone, while
calling for a next-generation vision language model to achieve Pareto-optimal
cost-efficiency.
MIRO: MultI-Reward cOnditioned pretraining improves T2I quality and
  efficiency
Authors: name: Nicolas Dufour, name: Lucas Degeorge, name: Arijit Ghosh, name: Vicky Kalogeiton, name: David Picard
Published: 2025-10-29 18:59:17

Current text-to-image generative models are trained on large uncurated
datasets to enable diverse generation capabilities. However, this does not
align well with user preferences. Recently, reward models have been
specifically designed to perform post-hoc selection of generated images and
align them to a reward, typically user preference. This discarding of
informative data together with the optimizing for a single reward tend to harm
diversity, semantic fidelity and efficiency. Instead of this post-processing,
we propose to condition the model on multiple reward models during training to
let the model learn user preferences directly. We show that this not only
dramatically improves the visual quality of the generated images but it also
significantly speeds up the training. Our proposed method, called MIRO,
achieves state-of-the-art performances on the GenEval compositional benchmark
and user-preference scores (PickAScore, ImageReward, HPSv2).
ChartAB: A Benchmark for Chart Grounding & Dense Alignment
Authors: name: Aniruddh Bansal, name: Davit Soselia, name: Dang Nguyen, name: Tianyi Zhou
Published: 2025-10-30 17:56:31

Charts play an important role in visualization, reasoning, data analysis, and
the exchange of ideas among humans. However, existing vision-language models
(VLMs) still lack accurate perception of details and struggle to extract
fine-grained structures from charts. Such limitations in chart grounding also
hinder their ability to compare multiple charts and reason over them. In this
paper, we introduce a novel "ChartAlign Benchmark (ChartAB)" to provide a
comprehensive evaluation of VLMs in chart grounding tasks, i.e., extracting
tabular data, localizing visualization elements, and recognizing various
attributes from charts of diverse types and complexities. We design a JSON
template to facilitate the calculation of evaluation metrics specifically
tailored for each grounding task. By incorporating a novel two-stage inference
workflow, the benchmark can further evaluate VLMs' capability to align and
compare elements/attributes across two charts. Our analysis of evaluations on
several recent VLMs reveals new insights into their perception biases,
weaknesses, robustness, and hallucinations in chart understanding. These
findings highlight the fine-grained discrepancies among VLMs in chart
understanding tasks and point to specific skills that need to be strengthened
in current models.