Orionnnn/hub-stats huggingface dataset:
Option-aware Temporally Abstracted Value for Offline Goal-Conditioned
  Reinforcement Learning
Authors: name: Hongjoon Ahn, name: Heewoong Choi, name: Jisu Han, name: Taesup Moon
Published: 2025-05-19 05:51:11

Offline goal-conditioned reinforcement learning (GCRL) offers a practical
learning paradigm where goal-reaching policies are trained from abundant
unlabeled (reward-free) datasets without additional environment interaction.
However, offline GCRL still struggles with long-horizon tasks, even with recent
advances that employ hierarchical policy structures, such as HIQL. By
identifying the root cause of this challenge, we observe the following
insights: First, performance bottlenecks mainly stem from the high-level
policy's inability to generate appropriate subgoals. Second, when learning the
high-level policy in the long-horizon regime, the sign of the advantage signal
frequently becomes incorrect. Thus, we argue that improving the value function
to produce a clear advantage signal for learning the high-level policy is
essential. In this paper, we propose a simple yet effective solution:
Option-aware Temporally Abstracted value learning, dubbed OTA, which
incorporates temporal abstraction into the temporal-difference learning
process. By modifying the value update to be option-aware, the proposed
learning scheme contracts the effective horizon length, enabling better
advantage estimates even in long-horizon regimes. We experimentally show that
the high-level policy extracted using the OTA value function achieves strong
performance on complex tasks from OGBench, a recently proposed offline GCRL
benchmark, including maze navigation and visual robotic manipulation
environments.
ModernGBERT: German-only 1B Encoder Model Trained from Scratch
Authors: name: Anton Ehrmanntraut, name: Julia Wunderle, name: Jan Pfister, name: Fotis Jannidis, name: Andreas Hotho
Published: 2025-05-19 14:07:20

Despite the prominence of decoder-only language models, encoders remain
crucial for resource-constrained applications. We introduce ModernGBERT (134M,
1B), a fully transparent family of German encoder models trained from scratch,
incorporating architectural innovations from ModernBERT. To evaluate the
practical trade-offs of training encoders from scratch, we also present
LL\"aMmlein2Vec (120M, 1B, 7B), a family of encoders derived from German
decoder-only models via LLM2Vec. We benchmark all models on natural language
understanding, text embedding, and long-context reasoning tasks, enabling a
controlled comparison between dedicated encoders and converted decoders. Our
results show that ModernGBERT 1B outperforms prior state-of-the-art German
encoders as well as encoders adapted via LLM2Vec, with regard to performance
and parameter-efficiency. All models, training data, checkpoints and code are
publicly available, advancing the German NLP ecosystem with transparent,
high-performance encoder models.
TAGS: A Test-Time Generalist-Specialist Framework with
  Retrieval-Augmented Reasoning and Verification
Authors: name: Jianghao Wu, name: Feilong Tang, name: Yulong Li, name: Ming Hu, name: Haochen Xue, name: Shoaib Jameel, name: Yutong Xie, name: Imran Razzak
Published: 2025-05-23 18:28:59

Recent advances such as Chain-of-Thought prompting have significantly
improved large language models (LLMs) in zero-shot medical reasoning. However,
prompting-based methods often remain shallow and unstable, while fine-tuned
medical LLMs suffer from poor generalization under distribution shifts and
limited adaptability to unseen clinical scenarios. To address these
limitations, we present TAGS, a test-time framework that combines a broadly
capable generalist with a domain-specific specialist to offer complementary
perspectives without any model fine-tuning or parameter updates. To support
this generalist-specialist reasoning process, we introduce two auxiliary
modules: a hierarchical retrieval mechanism that provides multi-scale exemplars
by selecting examples based on both semantic and rationale-level similarity,
and a reliability scorer that evaluates reasoning consistency to guide final
answer aggregation. TAGS achieves strong performance across nine MedQA
benchmarks, boosting GPT-4o accuracy by 13.8%, DeepSeek-R1 by 16.8%, and
improving a vanilla 7B model from 14.1% to 23.9%. These results surpass several
fine-tuned medical LLMs, without any parameter updates. The code will be
available at https://github.com/JianghaoWu/TAGS.
Accelerating Nash Learning from Human Feedback via Mirror Prox
Authors: name: Daniil Tiapkin, name: Daniele Calandriello, name: Denis Belomestny, name: Eric Moulines, name: Alexey Naumov, name: Kashif Rasul, name: Michal Valko, name: Pierre Menard
Published: 2025-05-26 09:17:32

Traditional Reinforcement Learning from Human Feedback (RLHF) often relies on
reward models, frequently assuming preference structures like the Bradley-Terry
model, which may not accurately capture the complexities of real human
preferences (e.g., intransitivity). Nash Learning from Human Feedback (NLHF)
offers a more direct alternative by framing the problem as finding a Nash
equilibrium of a game defined by these preferences. In this work, we introduce
Nash Mirror Prox (Nash-MP), an online NLHF algorithm that leverages
the Mirror Prox optimization scheme to achieve fast and stable convergence to
the Nash equilibrium. Our theoretical analysis establishes that Nash-MP
exhibits last-iterate linear convergence towards the beta-regularized Nash
equilibrium. Specifically, we prove that the KL-divergence to the optimal
policy decreases at a rate of order (1+2beta)^{-N/2}, where N is a number
of preference queries. We further demonstrate last-iterate linear convergence
for the exploitability gap and uniformly for the span semi-norm of
log-probabilities, with all these rates being independent of the size of the
action space. Furthermore, we propose and analyze an approximate version of
Nash-MP where proximal steps are estimated using stochastic policy gradients,
making the algorithm closer to applications. Finally, we detail a practical
implementation strategy for fine-tuning large language models and present
experiments that demonstrate its competitive performance and compatibility with
existing methods.
Alchemist: Turning Public Text-to-Image Data into Generative Gold
Authors: name: Valerii Startsev, name: Alexander Ustyuzhanin, name: Alexey Kirillov, name: Dmitry Baranchuk, name: Sergey Kastryulin
Published: 2025-05-25 20:08:20

Pre-training equips text-to-image (T2I) models with broad world knowledge,
but this alone is often insufficient to achieve high aesthetic quality and
alignment. Consequently, supervised fine-tuning (SFT) is crucial for further
refinement. However, its effectiveness highly depends on the quality of the
fine-tuning dataset. Existing public SFT datasets frequently target narrow
domains (e.g., anime or specific art styles), and the creation of high-quality,
general-purpose SFT datasets remains a significant challenge. Current curation
methods are often costly and struggle to identify truly impactful samples. This
challenge is further complicated by the scarcity of public general-purpose
datasets, as leading models often rely on large, proprietary, and poorly
documented internal data, hindering broader research progress. This paper
introduces a novel methodology for creating general-purpose SFT datasets by
leveraging a pre-trained generative model as an estimator of high-impact
training samples. We apply this methodology to construct and release Alchemist,
a compact (3,350 samples) yet highly effective SFT dataset. Experiments
demonstrate that Alchemist substantially improves the generative quality of
five public T2I models while preserving diversity and style. Additionally, we
release the fine-tuned models' weights to the public.
Jodi: Unification of Visual Generation and Understanding via Joint
  Modeling
Authors: name: Yifeng Xu, name: Zhenliang He, name: Meina Kan, name: Shiguang Shan, name: Xilin Chen
Published: 2025-05-25 10:40:52

Visual generation and understanding are two deeply interconnected aspects of
human intelligence, yet they have been traditionally treated as separate tasks
in machine learning. In this paper, we propose Jodi, a diffusion framework that
unifies visual generation and understanding by jointly modeling the image
domain and multiple label domains. Specifically, Jodi is built upon a linear
diffusion transformer along with a role switch mechanism, which enables it to
perform three particular types of tasks: (1) joint generation, where the model
simultaneously generates images and multiple labels; (2) controllable
generation, where images are generated conditioned on any combination of
labels; and (3) image perception, where multiple labels can be predicted at
once from a given image. Furthermore, we present the Joint-1.6M dataset, which
contains 200,000 high-quality images collected from public sources, automatic
labels for 7 visual domains, and LLM-generated captions. Extensive experiments
demonstrate that Jodi excels in both generation and understanding tasks and
exhibits strong extensibility to a wider range of visual domains. Code is
available at https://github.com/VIPL-GENUN/Jodi.
MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research
Authors: name: Hui Chen, name: Miao Xiong, name: Yujie Lu, name: Wei Han, name: Ailin Deng, name: Yufei He, name: Jiaying Wu, name: Yibo Li, name: Yue Liu, name: Bryan Hooi
Published: 2025-05-26 13:18:37

Recent advancements in AI agents have demonstrated their growing potential to
drive and support scientific discovery. In this work, we introduce MLR-Bench, a
comprehensive benchmark for evaluating AI agents on open-ended machine learning
research. MLR-Bench includes three key components: (1) 201 research tasks
sourced from NeurIPS, ICLR, and ICML workshops covering diverse ML topics; (2)
MLR-Judge, an automated evaluation framework combining LLM-based reviewers with
carefully designed review rubrics to assess research quality; and (3)
MLR-Agent, a modular agent scaffold capable of completing research tasks
through four stages: idea generation, proposal formulation, experimentation,
and paper writing. Our framework supports both stepwise assessment across these
distinct research stages, and end-to-end evaluation of the final research
paper. We then use MLR-Bench to evaluate six frontier LLMs and an advanced
coding agent, finding that while LLMs are effective at generating coherent
ideas and well-structured papers, current coding agents frequently (e.g., in
80% of the cases) produce fabricated or invalidated experimental
results--posing a major barrier to scientific reliability. We validate
MLR-Judge through human evaluation, showing high agreement with expert
reviewers, supporting its potential as a scalable tool for research evaluation.
We open-source MLR-Bench to help the community benchmark, diagnose, and improve
AI research agents toward trustworthy and transparent scientific discovery.
An Embarrassingly Simple Defense Against LLM Abliteration Attacks
Authors: name: Harethah Abu Shairah, name: Hasan Abed Al Kader Hammoud, name: Bernard Ghanem, name: George Turkiyyah
Published: 2025-05-25 09:18:24

Large language models (LLMs) are typically aligned to comply with safety
guidelines by refusing harmful instructions. A recent attack, termed
abliteration, isolates and suppresses the single latent direction most
responsible for refusal behavior, enabling the model to generate unethical
content. We propose a defense that modifies how models generate refusals. We
construct an extended-refusal dataset that contains harmful prompts with a full
response that justifies the reason for refusal. We then fine-tune
Llama-2-7B-Chat and Qwen2.5-Instruct (1.5B and 3B parameters) on our
extended-refusal dataset, and evaluate the resulting systems on a set of
harmful prompts. In our experiments, extended-refusal models maintain high
refusal rates, dropping at most by 10%, whereas baseline models' refusal rates
drop by 70-80% after abliteration. A broad evaluation of safety and utility
shows that extended-refusal fine-tuning neutralizes the abliteration attack
while preserving general performance.
AdaCtrl: Towards Adaptive and Controllable Reasoning via
  Difficulty-Aware Budgeting
Authors: name: Shijue Huang, name: Hongru Wang, name: Wanjun Zhong, name: Zhaochen Su, name: Jiazhan Feng, name: Bowen Cao, name: Yi R. Fung
Published: 2025-05-24 18:46:50

Modern large reasoning models demonstrate impressive problem-solving
capabilities by employing sophisticated reasoning strategies. However, they
often struggle to balance efficiency and effectiveness, frequently generating
unnecessarily lengthy reasoning chains for simple problems. In this work, we
propose AdaCtrl, a novel framework to support both difficulty-aware adaptive
reasoning budget allocation and explicit user control over reasoning depth.
AdaCtrl dynamically adjusts its reasoning length based on self-assessed problem
difficulty, while also allowing users to manually control the budget to
prioritize either efficiency or effectiveness. This is achieved through a
two-stage training pipeline: an initial cold-start fine-tuning phase to instill
the ability to self-aware difficulty and adjust reasoning budget, followed by a
difficulty-aware reinforcement learning (RL) stage that refines the model's
adaptive reasoning strategies and calibrates its difficulty assessments based
on its evolving capabilities during online training. To enable intuitive user
interaction, we design explicit length-triggered tags that function as a
natural interface for budget control. Empirical results show that AdaCtrl
adapts reasoning length based on estimated difficulty, compared to the standard
training baseline that also incorporates fine-tuning and RL, it yields
performance improvements and simultaneously reduces response length by 10.06%
and 12.14% on the more challenging AIME2024 and AIME2025 datasets, which
require elaborate reasoning, and by 62.05% and 91.04% on the MATH500 and GSM8K
datasets, where more concise responses are sufficient. Furthermore, AdaCtrl
enables precise user control over the reasoning budget, allowing for tailored
responses to meet specific needs.
PATS: Process-Level Adaptive Thinking Mode Switching
Authors: name: Yi Wang, name: Junxiao Liu, name: Shimao Zhang, name: Jiajun Chen, name: Shujian Huang
Published: 2025-05-25 17:58:50

Current large-language models (LLMs) typically adopt a fixed reasoning
strategy, either simple or complex, for all questions, regardless of their
difficulty. This neglect of variation in task and reasoning process complexity
leads to an imbalance between performance and efficiency. Existing methods
attempt to implement training-free fast-slow thinking system switching to
handle problems of varying difficulty, but are limited by coarse-grained
solution-level strategy adjustments. To address this issue, we propose a novel
reasoning paradigm: Process-Level Adaptive Thinking Mode Switching (PATS),
which enables LLMs to dynamically adjust their reasoning strategy based on the
difficulty of each step, optimizing the balance between accuracy and
computational efficiency. Our approach integrates Process Reward Models (PRMs)
with Beam Search, incorporating progressive mode switching and bad-step penalty
mechanisms. Experiments on diverse mathematical benchmarks demonstrate that our
methodology achieves high accuracy while maintaining moderate token usage. This
study emphasizes the significance of process-level, difficulty-aware reasoning
strategy adaptation, offering valuable insights into efficient inference for
LLMs.
LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language
  Diffusion Models
Authors: name: Fengqi Zhu, name: Rongzhen Wang, name: Shen Nie, name: Xiaolu Zhang, name: Chunwei Wu, name: Jun Hu, name: Jun Zhou, name: Jianfei Chen, name: Yankai Lin, name: Ji-Rong Wen, name: Chongxuan Li
Published: 2025-05-25 16:36:20

While Masked Diffusion Models (MDMs), such as LLaDA, present a promising
paradigm for language modeling, there has been relatively little effort in
aligning these models with human preferences via reinforcement learning. The
challenge primarily arises from the high variance in Evidence Lower Bound
(ELBO)-based likelihood estimates required for preference optimization. To
address this issue, we propose Variance-Reduced Preference Optimization (VRPO),
a framework that formally analyzes the variance of ELBO estimators and derives
bounds on both the bias and variance of preference optimization gradients.
Building on this theoretical foundation, we introduce unbiased variance
reduction strategies, including optimal Monte Carlo budget allocation and
antithetic sampling, that significantly improve the performance of MDM
alignment. We demonstrate the effectiveness of VRPO by applying it to LLaDA,
and the resulting model, LLaDA 1.5, outperforms its SFT-only predecessor
consistently and significantly across mathematical (GSM8K +4.7), code
(HumanEval +3.0, MBPP +1.8), and alignment benchmarks (IFEval +4.0, Arena-Hard
+4.3). Furthermore, LLaDA 1.5 demonstrates a highly competitive mathematical
performance compared to strong language MDMs and ARMs. Project page:
https://ml-gsai.github.io/LLaDA-1.5-Demo/.
MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs
Authors: name: Zaid Alyafeai, name: Maged S. Al-Shaibani, name: Bernard Ghanem
Published: 2025-05-26 10:31:26

Metadata extraction is essential for cataloging and preserving datasets,
enabling effective research discovery and reproducibility, especially given the
current exponential growth in scientific research. While Masader (Alyafeai et
al.,2021) laid the groundwork for extracting a wide range of metadata
attributes from Arabic NLP datasets' scholarly articles, it relies heavily on
manual annotation. In this paper, we present MOLE, a framework that leverages
Large Language Models (LLMs) to automatically extract metadata attributes from
scientific papers covering datasets of languages other than Arabic. Our
schema-driven methodology processes entire documents across multiple input
formats and incorporates robust validation mechanisms for consistent output.
Additionally, we introduce a new benchmark to evaluate the research progress on
this task. Through systematic analysis of context length, few-shot learning,
and web browsing integration, we demonstrate that modern LLMs show promising
results in automating this task, highlighting the need for further future work
improvements to ensure consistent and reliable performance. We release the
code: https://github.com/IVUL-KAUST/MOLE and dataset:
https://huggingface.co/datasets/IVUL-KAUST/MOLE for the research community.
Mutarjim: Advancing Bidirectional Arabic-English Translation with a
  Small Language Model
Authors: name: Khalil Hennara, name: Muhammad Hreden, name: Mohamed Motaism Hamed, name: Zeina Aldallal, name: Sara Chrouf, name: Safwan AlModhayan
Published: 2025-05-23 13:42:21

We introduce Mutarjim, a compact yet powerful language model for
bidirectional Arabic-English translation. While large-scale LLMs have shown
impressive progress in natural language processing tasks, including machine
translation, smaller models. Leveraging this insight, we developed Mutarjim
based on Kuwain-1.5B , a language model tailored for both Arabic and English.
Despite its modest size, Mutarjim outperforms much larger models on several
established benchmarks, achieved through an optimized two-phase training
approach and a carefully curated, high-quality training corpus.. Experimental
results show that Mutarjim rivals models up to 20 times larger while
significantly reducing computational costs and training requirements. We also
introduce Tarjama-25, a new benchmark designed to overcome limitations in
existing Arabic-English benchmarking datasets, such as domain narrowness, short
sentence lengths, and English-source bias. Tarjama-25 comprises 5,000
expert-reviewed sentence pairs and spans a wide range of domains, offering a
more comprehensive and balanced evaluation framework. Notably, Mutarjim
achieves state-of-the-art performance on the English-to-Arabic task in
Tarjama-25, surpassing even significantly larger and proprietary models like
GPT-4o mini. We publicly release Tarjama-25 to support future research and
advance the evaluation of Arabic-English translation systems.
GLEAM: Learning Generalizable Exploration Policy for Active Mapping in
  Complex 3D Indoor Scenes
Authors: name: Xiao Chen, name: Tai Wang, name: Quanyi Li, name: Tao Huang, name: Jiangmiao Pang, name: Tianfan Xue
Published: 2025-05-26 17:59:52

Generalizable active mapping in complex unknown environments remains a
critical challenge for mobile robots. Existing methods, constrained by
insufficient training data and conservative exploration strategies, exhibit
limited generalizability across scenes with diverse layouts and complex
connectivity. To enable scalable training and reliable evaluation, we introduce
GLEAM-Bench, the first large-scale benchmark designed for generalizable active
mapping with 1,152 diverse 3D scenes from synthetic and real-scan datasets.
Building upon this foundation, we propose GLEAM, a unified generalizable
exploration policy for active mapping. Its superior generalizability comes
mainly from our semantic representations, long-term navigable goals, and
randomized strategies. It significantly outperforms state-of-the-art methods,
achieving 66.50% coverage (+9.49%) with efficient trajectories and improved
mapping accuracy on 128 unseen complex scenes. Project page:
https://xiao-chen.tech/gleam/.
StructEval: Benchmarking LLMs' Capabilities to Generate Structural
  Outputs
Authors: name: Jialin Yang, name: Dongfu Jiang, name: Lipeng He, name: Sherman Siu, name: Yuxuan Zhang, name: Disen Liao, name: Zhuofeng Li, name: Huaye Zeng, name: Yiming Jia, name: Haozhe Wang, name: Benjamin Schneider, name: Chi Ruan, name: Wentao Ma, name: Zhiheng Lyu, name: Yifei Wang, name: Yi Lu, name: Quy Duc Do, name: Ziyan Jiang, name: Ping Nie, name: Wenhu Chen
Published: 2025-05-26 15:40:42

As Large Language Models (LLMs) become integral to software development
workflows, their ability to generate structured outputs has become critically
important. We introduce StructEval, a comprehensive benchmark for evaluating
LLMs' capabilities in producing both non-renderable (JSON, YAML, CSV) and
renderable (HTML, React, SVG) structured formats. Unlike prior benchmarks,
StructEval systematically evaluates structural fidelity across diverse formats
through two paradigms: 1) generation tasks, producing structured output from
natural language prompts, and 2) conversion tasks, translating between
structured formats. Our benchmark encompasses 18 formats and 44 types of task,
with novel metrics for format adherence and structural correctness. Results
reveal significant performance gaps, even state-of-the-art models like o1-mini
achieve only 75.58 average score, with open-source alternatives lagging
approximately 10 points behind. We find generation tasks more challenging than
conversion tasks, and producing correct visual content more difficult than
generating text-only structures.