Orionnnn/hub-stats huggingface dataset:
AION-1: Omnimodal Foundation Model for Astronomical Sciences
Authors: name: Liam Parker, name: Francois Lanusse, name: Jeff Shen, name: Ollie Liu, name: Tom Hehir, name: Leopoldo Sarra, name: Lucas Meyer, name: Micah Bowles, name: Sebastian Wagner-Carena, name: Helen Qu, name: Siavash Golkar, name: Alberto Bietti, name: Hatim Bourfoune, name: Nathan Casserau, name: Pierre Cornette, name: Keiya Hirashima, name: Geraud Krawezik, name: Ruben Ohana, name: Nicholas Lourie, name: Michael McCabe, name: Rudy Morel, name: Payel Mukhopadhyay, name: Mariel Pettee, name: Bruno Regaldo-Saint Blancard, name: Kyunghyun Cho, name: Miles Cranmer, name: Shirley Ho
Published: 2025-10-20 18:00:00

While foundation models have shown promise across a variety of fields,
astronomy still lacks a unified framework for joint modeling across its highly
diverse data modalities. In this paper, we present AION-1, a family of
large-scale multimodal foundation models for astronomy. AION-1 integrates
heterogeneous imaging, spectroscopic, and scalar data using a two-stage
architecture: modality-specific tokenization followed by transformer-based
masked modeling of cross-modal token sequences. The model is pretrained on five
large-scale surveys: Legacy Survey, Hyper Suprime-Cam (HSC), Sloan Digital Sky
Survey (SDSS), Dark Energy Spectroscopic Instrument (DESI), and Gaia. These
span more than 200 million observations of stars, galaxies, and quasars. With a
single frozen encoder, AION-1 achieves strong results on a broad suite of
downstream tasks, including galaxy and stellar property estimation, galaxy
morphology classification, similarity-based retrieval, galaxy image
segmentation, and spectral super-resolution. We release AION-1 model variants
ranging from 300 M to 3.1 B parameters. Beyond astronomy, AION-1 provides a
scalable blueprint for multimodal scientific foundation models that can
seamlessly integrate noisy, instrument-specific observations. All code,
tokenizers, pretrained weights, and a lightweight evaluation suite are released
under an open-source license.
MoReBench: Evaluating Procedural and Pluralistic Moral Reasoning in
  Language Models, More than Outcomes
Authors: name: Yu Ying Chiu, name: Michael S. Lee, name: Rachel Calcott, name: Brandon Handoko, name: Paul de Font-Reaulx, name: Paula Rodriguez, name: Chen Bo Calvin Zhang, name: Ziwen Han, name: Udari Madhushani Sehwag, name: Yash Maurya, name: Christina Q Knight, name: Harry R. Lloyd, name: Florence Bacus, name: Mantas Mazeika, name: Bing Liu, name: Yejin Choi, name: Mitchell L Gordon, name: Sydney Levine
Published: 2025-10-18 07:34:31

As AI systems progress, we rely more on them to make decisions with us and
for us. To ensure that such decisions are aligned with human values, it is
imperative for us to understand not only what decisions they make but also how
they come to those decisions. Reasoning language models, which provide both
final responses and (partially transparent) intermediate thinking traces,
present a timely opportunity to study AI procedural reasoning. Unlike math and
code problems which often have objectively correct answers, moral dilemmas are
an excellent testbed for process-focused evaluation because they allow for
multiple defensible conclusions. To do so, we present MoReBench: 1,000 moral
scenarios, each paired with a set of rubric criteria that experts consider
essential to include (or avoid) when reasoning about the scenarios. MoReBench
contains over 23 thousand criteria including identifying moral considerations,
weighing trade-offs, and giving actionable recommendations to cover cases on AI
advising humans moral decisions as well as making moral decisions autonomously.
Separately, we curate MoReBench-Theory: 150 examples to test whether AI can
reason under five major frameworks in normative ethics. Our results show that
scaling laws and existing benchmarks on math, code, and scientific reasoning
tasks fail to predict models' abilities to perform moral reasoning. Models also
show partiality towards specific moral frameworks (e.g., Benthamite Act
Utilitarianism and Kantian Deontology), which might be side effects of popular
training paradigms. Together, these benchmarks advance process-focused
reasoning evaluation towards safer and more transparent AI.
TrajSelector: Harnessing Latent Representations for Efficient and
  Effective Best-of-N in Large Reasoning Model
Authors: name: Bin Yu, name: Xinming Wang, name: Shijie Lian, name: Haotian Li, name: Changti Wu, name: Ruina Hu, name: Bailing Wang, name: Yuliang Wei, name: Kai Chen
Published: 2025-10-18 11:01:39

Large language models (LLMs) have shown remarkable progress in complex
reasoning tasks, largely enabled by test-time scaling (TTS) paradigms that
allocate additional compute during inference. Among these, external TTS
(particularly the Best-of-N selection paradigm) yields scalable performance
improvements by selecting from multiple independently generated reasoning
trajectories. However, this approach faces key limitations: (i) the high
computational overhead of deploying process reward models, (ii) the
underutilization of the LLM's intrinsic latent representations. We introduce
TrajSelector, an efficient and effective Best-of-N framework that exploit the
hidden states in the sampler LLM for process-level scoring. A lightweight
verifier (with only 0.6B parameters) evaluates the quality of step-wise
trajectory, and then aggregates these scores to identify the optimal reasoning
trajectory. Our framework employs a fully data-driven, end-to-end training
recipe that eliminates reliance on massive step-level annotations. Experiential
results across five benchmarks demonstrate that TrajSelector delivers
consistent performance gains. In Best-of-32 settings, it surpasses majority
voting by 4.61% accuracy and outperforms existing process reward models by
4.31% to 12.21%, all while maintaining lower inference costs.
What Limits Agentic Systems Efficiency?
Authors: name: Song Bian, name: Minghao Yan, name: Anand Jayarajan, name: Gennady Pekhimenko, name: Shivaram Venkataraman
Published: 2025-10-18 00:21:45

Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have
demonstrated strong reasoning capabilities. To further enhance LLM
capabilities, recent agentic systems, such as Deep Research, incorporate web
interactions into LLM reasoning to mitigate uncertainties and reduce potential
errors. However, existing research predominantly focuses on reasoning
performance, often neglecting the efficiency of agentic systems. In this work,
we present a comprehensive empirical study that identifies efficiency
bottlenecks in web-interactive agentic systems. We decompose end-to-end latency
into two primary components: LLM API latency and web environment latency. We
conduct a comprehensive empirical study across 15 models and 5 providers to
demonstrate high variability in API-based agentic systems. We observe that web
environment latency can contribute as much as 53.7% to the overall latency in a
web-based agentic system. To improve latency, we propose SpecCache, a caching
framework augmented with speculative execution that can reduce web environment
overhead. Extensive evaluations on two standard benchmarks show that our
approach improves the cache hit rate by up to 58x compared to a random caching
strategy, while reducing web environment overhead by up to 3.2x, without
degrading agentic system performance.
Automated Composition of Agents: A Knapsack Approach for Agentic
  Component Selection
Authors: name: Michelle Yuan, name: Khushbu Pahwa, name: Shuaichen Chang, name: Mustafa Kaba, name: Jiarong Jiang, name: Xiaofei Ma, name: Yi Zhang, name: Monica Sunkara
Published: 2025-10-18 13:37:47

Designing effective agentic systems requires the seamless composition and
integration of agents, tools, and models within dynamic and uncertain
environments. Most existing methods rely on static, semantic retrieval
approaches for tool or agent discovery. However, effective reuse and
composition of existing components remain challenging due to incomplete
capability descriptions and the limitations of retrieval methods. Component
selection suffers because the decisions are not based on capability, cost, and
real-time utility. To address these challenges, we introduce a structured,
automated framework for agentic system composition that is inspired by the
knapsack problem. Our framework enables a composer agent to systematically
identify, select, and assemble an optimal set of agentic components by jointly
considering performance, budget constraints, and compatibility. By dynamically
testing candidate components and modeling their utility in real-time, our
approach streamlines the assembly of agentic systems and facilitates scalable
reuse of resources. Empirical evaluation with Claude 3.5 Sonnet across five
benchmarking datasets shows that our online-knapsack-based composer
consistently lies on the Pareto frontier, achieving higher success rates at
significantly lower component costs compared to our baselines. In the
single-agent setup, the online knapsack composer shows a success rate
improvement of up to 31.6% in comparison to the retrieval baselines. In
multi-agent systems, the online knapsack composer increases success rate from
37% to 87% when agents are selected from an agent inventory of 100+ agents. The
substantial performance gap confirms the robust adaptability of our method
across diverse domains and budget constraints.
Beacon: Single-Turn Diagnosis and Mitigation of Latent Sycophancy in
  Large Language Models
Authors: name: Sanskar Pandey, name: Ruhaan Chopra, name: Angkul Puniya, name: Sohom Pal
Published: 2025-10-19 06:36:57

Large language models internalize a structural trade-off between truthfulness
and obsequious flattery, emerging from reward optimization that conflates
helpfulness with polite submission. This latent bias, known as sycophancy,
manifests as a preference for user agreement over principled reasoning. We
introduce Beacon, a single-turn forced-choice benchmark that isolates this bias
independent of conversational context, enabling precise measurement of the
tension between factual accuracy and submissive bias. Evaluations across twelve
state-of-the-art models reveal that sycophancy decomposes into stable
linguistic and affective sub-biases, each scaling with model capacity. We
further propose prompt-level and activation-level interventions that modulate
these biases in opposing directions, exposing the internal geometry of
alignment as a dynamic manifold between truthfulness and socially compliant
judgment. Beacon reframes sycophancy as a measurable form of normative
misgeneralization, providing a reproducible foundation for studying and
mitigating alignment drift in large-scale generative systems.
Test-Time Scaling of Reasoning Models for Machine Translation
Authors: name: Zihao Li, name: Shaoxiong Ji, name: Jörg Tiedemann
Published: 2025-10-07 21:15:18

Test-time scaling (TTS) has enhanced the performance of Reasoning Models
(RMs) on various tasks such as math and coding, yet its efficacy in machine
translation (MT) remains underexplored. This paper investigates whether
increased inference-time computation improves translation quality. We evaluate
12 RMs across a diverse suite of MT benchmarks spanning multiple domains,
examining three scenarios: direct translation, forced-reasoning extrapolation,
and post-editing. Our findings show that for general-purpose RMs, TTS provides
limited and inconsistent benefits for direct translation, with performance
quickly plateauing. However, the effectiveness of TTS is unlocked by
domain-specific fine-tuning, which aligns a model's reasoning process with task
requirements, leading to consistent improvements up to an optimal,
self-determined reasoning depth. We also find that forcing a model to reason
beyond its natural stopping point consistently degrades translation quality. In
contrast, TTS proves highly effective in a post-editing context, reliably
turning self-correction into a beneficial process. These results indicate that
the value of inference-time computation in MT lies not in enhancing single-pass
translation with general models, but in targeted applications like multi-step,
self-correction workflows and in conjunction with task-specialized models.
QueST: Incentivizing LLMs to Generate Difficult Problems
Authors: name: Hanxu Hu, name: Xingxing Zhang, name: Jannis Vamvas, name: Rico Sennrich, name: Furu Wei
Published: 2025-10-20 16:29:53

Large Language Models have achieved strong performance on reasoning tasks,
solving competition-level coding and math problems. However, their scalability
is limited by human-labeled datasets and the lack of large-scale, challenging
coding problem training data. Existing competitive coding datasets contain only
thousands to tens of thousands of problems. Previous synthetic data generation
methods rely on either augmenting existing instruction datasets or selecting
challenging problems from human-labeled data. In this paper, we propose QueST,
a novel framework which combines difficulty-aware graph sampling and
difficulty-aware rejection fine-tuning that directly optimizes specialized
generators to create challenging coding problems. Our trained generators
demonstrate superior capability compared to even GPT-4o at creating challenging
problems that benefit downstream performance. We leverage QueST to generate
large-scale synthetic coding problems, which we then use to distill from strong
teacher models with long chain-of-thought or to conduct reinforcement learning
for smaller models, proving effective in both scenarios. Our distillation
experiments demonstrate significant performance gains. Specifically, after
fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we
surpass the performance of the original Qwen3-8B on LiveCodeBench. With an
additional 112K examples (i.e., 28K human-written problems paired with multiple
synthetic solutions), our 8B model matches the performance of the much larger
DeepSeek-R1-671B. These findings indicate that generating complex problems via
QueST offers an effective and scalable approach to advancing the frontiers of
competitive coding and reasoning for large language models.
Agentic Reinforcement Learning for Search is Unsafe
Authors: name: Yushi Yang, name: Shreyansh Padarha, name: Andrew Lee, name: Adam Mahdi
Published: 2025-10-20 11:19:37

Agentic reinforcement learning (RL) trains large language models to
autonomously call tools during reasoning, with search as the most common
application. These models excel at multi-step reasoning tasks, but their safety
properties are not well understood. In this study, we show that RL-trained
search models inherit refusal from instruction tuning and often deflect harmful
requests by turning them into safe queries. However, this safety is fragile.
Two simple attacks, one that forces the model to begin response with search
(Search attack), another that encourages models to repeatedly search
(Multi-search attack), trigger cascades of harmful searches and answers. Across
two model families (Qwen, Llama) with both local and web search, these attacks
lower refusal rates by up to 60.0%, answer safety by 82.5%, and search-query
safety by 82.4%. The attacks succeed by triggering models to generate harmful,
request-mirroring search queries before they can generate the inherited refusal
tokens. This exposes a core weakness of current RL training: it rewards
continued generation of effective queries without accounting for their
harmfulness. As a result, RL search models have vulnerabilities that users can
easily exploit, making it urgent to develop safety-aware agentic RL pipelines
optimising for safe search.
On Non-interactive Evaluation of Animal Communication Translators
Authors: name: Orr Paradise, name: David F. Gruber, name: Adam Tauman Kalai
Published: 2025-10-17 15:56:30

If you had an AI Whale-to-English translator, how could you validate whether
or not it is working? Does one need to interact with the animals or rely on
grounded observations such as temperature? We provide theoretical and
proof-of-concept experimental evidence suggesting that interaction and even
observations may not be necessary for sufficiently complex languages. One may
be able to evaluate translators solely by their English outputs, offering
potential advantages in terms of safety, ethics, and cost. This is an instance
of machine translation quality evaluation (MTQE) without any reference
translations available. A key challenge is identifying ``hallucinations,''
false translations which may appear fluent and plausible. We propose using
segment-by-segment translation together with the classic NLP shuffle test to
evaluate translators. The idea is to translate animal communication, turn by
turn, and evaluate how often the resulting translations make more sense in
order than permuted. Proof-of-concept experiments on data-scarce human
languages and constructed languages demonstrate the potential utility of this
evaluation methodology. These human-language experiments serve solely to
validate our reference-free metric under data scarcity. It is found to
correlate highly with a standard evaluation based on reference translations,
which are available in our experiments. We also perform a theoretical analysis
suggesting that interaction may not be necessary nor efficient in the early
stages of learning to translate.
MultiVerse: A Multi-Turn Conversation Benchmark for Evaluating Large
  Vision and Language Models
Authors: name: Young-Jun Lee, name: Byung-Kwan Lee, name: Jianshu Zhang, name: Yechan Hwang, name: Byungsoo Ko, name: Han-Gyu Kim, name: Dongyu Yao, name: Xuankun Rong, name: Eojin Joo, name: Seung-Ho Han, name: Bowon Ko, name: Ho-Jin Choi
Published: 2025-10-18 21:00:12

Vision-and-Language Models (VLMs) have shown impressive capabilities on
single-turn benchmarks, yet real-world applications often demand more intricate
multi-turn dialogues. Existing multi-turn datasets (e.g, MMDU, ConvBench) only
partially capture the breadth and depth of conversational scenarios encountered
by users. In this work, we introduce MultiVerse, a novel multi-turn
conversation benchmark featuring 647 dialogues - each averaging four turns -
derived from a diverse set of 12 popular VLM evaluation benchmarks. With 484
tasks and 484 interaction goals, MultiVerse covers a wide range of topics, from
factual knowledge and perception to advanced reasoning tasks such as
mathematics and coding. To facilitate robust assessment, we propose a
checklist-based evaluation method that leverages GPT-4o as the automated
evaluator, measuring performance across 37 key aspects, including perceptual
accuracy, linguistic clarity, and factual correctness. We evaluate 18 VLMs on
MultiVerse, revealing that even the strongest models (e.g., GPT-4o) achieve
only a 50% success rate in complex multi-turn conversations, highlighting the
dataset's challenging nature. Notably, we find that providing full dialogue
context significantly enhances performance for smaller or weaker models,
emphasizing the importance of in-context learning. We believe MultiVerse is a
landscape of evaluating multi-turn interaction abilities for VLMs.
GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer
Authors: name: Sayan Deb Sarkar, name: Sinisa Stekovic, name: Vincent Lepetit, name: Iro Armeni
Published: 2025-10-17 18:22:04

Transferring appearance to 3D assets using different representations of the
appearance object - such as images or text - has garnered interest due to its
wide range of applications in industries like gaming, augmented reality, and
digital content creation. However, state-of-the-art methods still fail when the
geometry between the input and appearance objects is significantly different. A
straightforward approach is to directly apply a 3D generative model, but we
show that this ultimately fails to produce appealing results. Instead, we
propose a principled approach inspired by universal guidance. Given a
pretrained rectified flow model conditioned on image or text, our training-free
method interacts with the sampling process by periodically adding guidance.
This guidance can be modeled as a differentiable loss function, and we
experiment with two different types of guidance including part-aware losses for
appearance and self-similarity. Our experiments show that our approach
successfully transfers texture and geometric details to the input 3D asset,
outperforming baselines both qualitatively and quantitatively. We also show
that traditional metrics are not suitable for evaluating the task due to their
inability of focusing on local details and comparing dissimilar inputs, in
absence of ground truth data. We thus evaluate appearance transfer quality with
a GPT-based system objectively ranking outputs, ensuring robust and human-like
assessment, as further confirmed by our user study. Beyond showcased scenarios,
our method is general and could be extended to different types of diffusion
models and guidance functions.
Chronos-2: From Univariate to Universal Forecasting
Authors: name: Abdul Fatir Ansari, name: Oleksandr Shchur, name: Jaris Küken, name: Andreas Auer, name: Boran Han, name: Pedro Mercado, name: Syama Sundar Rangapuram, name: Huibin Shen, name: Lorenzo Stella, name: Xiyuan Zhang, name: Mononito Goswami, name: Shubham Kapoor, name: Danielle C. Maddix, name: Pablo Guerron, name: Tony Hu, name: Junming Yin, name: Nick Erickson, name: Prateek Mutalik Desai, name: Hao Wang, name: Huzefa Rangwala, name: George Karypis, name: Yuyang Wang, name: Michael Bohlke-Schneider
Published: 2025-10-17 17:00:53

Pretrained time series models have enabled inference-only forecasting systems
that produce accurate predictions without task-specific training. However,
existing approaches largely focus on univariate forecasting, limiting their
applicability in real-world scenarios where multivariate data and covariates
play a crucial role. We present Chronos-2, a pretrained model capable of
handling univariate, multivariate, and covariate-informed forecasting tasks in
a zero-shot manner. Chronos-2 employs a group attention mechanism that
facilitates in-context learning (ICL) through efficient information sharing
across multiple time series within a group, which may represent sets of related
series, variates of a multivariate series, or targets and covariates in a
forecasting task. These general capabilities are achieved through training on
synthetic datasets that impose diverse multivariate structures on univariate
series. Chronos-2 delivers state-of-the-art performance across three
comprehensive benchmarks: fev-bench, GIFT-Eval, and Chronos Benchmark II. On
fev-bench, which emphasizes multivariate and covariate-informed forecasting,
Chronos-2's universal ICL capabilities lead to substantial improvements over
existing models. On tasks involving covariates, it consistently outperforms
baselines by a wide margin. Case studies in the energy and retail domains
further highlight its practical advantages. The in-context learning
capabilities of Chronos-2 establish it as a general-purpose forecasting model
that can be used "as is" in real-world forecasting pipelines.