Orionnnn/hub-stats huggingface dataset:
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Authors: name: Jenny Zhang, name: Shengran Hu, name: Cong Lu, name: Robert Lange, name: Jeff Clune
Published: 2025-05-29 00:26:15

Today's AI systems have human-designed, fixed architectures and cannot
autonomously and continuously improve themselves. The advance of AI could
itself be automated. If done safely, that would accelerate AI development and
allow us to reap its benefits much sooner. Meta-learning can automate the
discovery of novel algorithms, but is limited by first-order improvements and
the human design of a suitable search space. The G\"odel machine proposed a
theoretical alternative: a self-improving AI that repeatedly modifies itself in
a provably beneficial manner. Unfortunately, proving that most changes are net
beneficial is impossible in practice. We introduce the Darwin G\"odel Machine
(DGM), a self-improving system that iteratively modifies its own code (thereby
also improving its ability to modify its own codebase) and empirically
validates each change using coding benchmarks. Inspired by Darwinian evolution
and open-endedness research, the DGM maintains an archive of generated coding
agents. It grows the archive by sampling an agent from it and using a
foundation model to create a new, interesting, version of the sampled agent.
This open-ended exploration forms a growing tree of diverse, high-quality
agents and allows the parallel exploration of many different paths through the
search space. Empirically, the DGM automatically improves its coding
capabilities (e.g., better code editing tools, long-context window management,
peer-review mechanisms), increasing performance on SWE-bench from 20.0% to
50.0%, and on Polyglot from 14.2% to 30.7%. Furthermore, the DGM significantly
outperforms baselines without self-improvement or open-ended exploration. All
experiments were done with safety precautions (e.g., sandboxing, human
oversight). The DGM is a significant step toward self-improving AI, capable of
gathering its own stepping stones along paths that unfold into endless
innovation.
SealQA: Raising the Bar for Reasoning in Search-Augmented Language
  Models
Authors: name: Thinh Pham, name: Nguyen Nguyen, name: Pratibha Zunjare, name: Weiyuan Chen, name: Yu-Min Tseng, name: Tu Vu
Published: 2025-06-01 16:04:34

We introduce SealQA, a new challenge benchmark for evaluating
SEarch-Augmented Language models on fact-seeking questions where web search
yields conflicting, noisy, or unhelpful results. SealQA comes in three flavors:
(1) Seal-0 (main) and (2) Seal-Hard, which assess factual accuracy and
reasoning capabilities, with Seal-0 focusing on the most challenging questions
where chat models (e.g., GPT-4.1) typically achieve near-zero accuracy; and (3)
LongSeal, which extends SealQA to test long-context, multi-document reasoning
in "needle-in-a-haystack" settings. Our evaluation reveals critical limitations
in current models: Even frontier LLMs perform poorly across all SealQA flavors.
On Seal-0, frontier agentic models equipped with tools like o3 and o4-mini
achieve only 17.1% and 6.3% accuracy, respectively, at their best reasoning
efforts. We find that advanced reasoning models such as DeepSeek-R1-671B and
o3-mini are highly vulnerable to noisy search results. Notably, increasing
test-time compute does not yield reliable gains across o3-mini, o4-mini, and
o3, with performance often plateauing or even declining early. Additionally,
while recent models are less affected by the "lost-in-the-middle" issue, they
still fail to reliably identify relevant documents in LongSeal when faced with
numerous distractors. To facilitate future work, we release SealQA at
huggingface.co/datasets/vtllms/sealqa.
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented
  Generation Systems
Authors: name: Yixiao Zeng, name: Tianyu Cao, name: Danqing Wang, name: Xinran Zhao, name: Zimeng Qiu, name: Morteza Ziyadi, name: Tongshuang Wu, name: Lei Li
Published: 2025-06-01 02:42:36

Retrieval-Augmented Generation (RAG) enhances recency and factuality in
answers. However, existing evaluations rarely test how well these systems cope
with real-world noise, conflicting between internal and external retrieved
contexts, or fast-changing facts. We introduce Retrieval-Aware Robustness
Evaluation (RARE), a unified framework and large-scale benchmark that jointly
stress-tests query and document perturbations over dynamic, time-sensitive
corpora. One of the central features of RARE is a knowledge-graph-driven
synthesis pipeline (RARE-Get) that automatically extracts single and multi-hop
relations from the customized corpus and generates multi-level question sets
without manual intervention. Leveraging this pipeline, we construct a dataset
(RARE-Set) spanning 400 expert-level time-sensitive finance, economics, and
policy documents and 48,322 questions whose distribution evolves as the
underlying sources change. To quantify resilience, we formalize
retrieval-conditioned robustness metrics (RARE-Met) that capture a model's
ability to remain correct or recover when queries, documents, or real-world
retrieval results are systematically altered. Our results show that RAG systems
exhibit surprising vulnerability to perturbations, with document robustness
consistently being the weakest point regardless of generator size or
architecture. RAG systems consistently show lower robustness on multi-hop
queries than single-hop queries across all domains.
Shuffle PatchMix Augmentation with Confidence-Margin Weighted
  Pseudo-Labels for Enhanced Source-Free Domain Adaptation
Authors: name: Prasanna Reddy Pulakurthi, name: Majid Rabbani, name: Jamison Heard, name: Sohail Dianat, name: Celso M. de Melo, name: Raghuveer Rao
Published: 2025-05-30 05:02:42

This work investigates Source-Free Domain Adaptation (SFDA), where a model
adapts to a target domain without access to source data. A new augmentation
technique, Shuffle PatchMix (SPM), and a novel reweighting strategy are
introduced to enhance performance. SPM shuffles and blends image patches to
generate diverse and challenging augmentations, while the reweighting strategy
prioritizes reliable pseudo-labels to mitigate label noise. These techniques
are particularly effective on smaller datasets like PACS, where overfitting and
pseudo-label noise pose greater risks. State-of-the-art results are achieved on
three major benchmarks: PACS, VisDA-C, and DomainNet-126. Notably, on PACS,
improvements of 7.3% (79.4% to 86.7%) and 7.2% are observed in single-target
and multi-target settings, respectively, while gains of 2.8% and 0.7% are
attained on DomainNet-126 and VisDA-C. This combination of advanced
augmentation and robust pseudo-label reweighting establishes a new benchmark
for SFDA. The code is available at: https://github.com/PrasannaPulakurthi/SPM
IVY-FAKE: A Unified Explainable Framework and Benchmark for Image and
  Video AIGC Detection
Authors: name: Wayne Zhang, name: Changjiang Jiang, name: Zhonghao Zhang, name: Chenyang Si, name: Fengchang Yu, name: Wei Peng
Published: 2025-06-01 12:20:22

The rapid advancement of Artificial Intelligence Generated Content (AIGC) in
visual domains has resulted in highly realistic synthetic images and videos,
driven by sophisticated generative frameworks such as diffusion-based
architectures. While these breakthroughs open substantial opportunities, they
simultaneously raise critical concerns about content authenticity and
integrity. Many current AIGC detection methods operate as black-box binary
classifiers, which offer limited interpretability, and no approach supports
detecting both images and videos in a unified framework. This dual limitation
compromises model transparency, reduces trustworthiness, and hinders practical
deployment. To address these challenges, we introduce IVY-FAKE , a novel,
unified, and large-scale dataset specifically designed for explainable
multimodal AIGC detection. Unlike prior benchmarks, which suffer from
fragmented modality coverage and sparse annotations, IVY-FAKE contains over
150,000 richly annotated training samples (images and videos) and 18,700
evaluation examples, each accompanied by detailed natural-language reasoning
beyond simple binary labels. Building on this, we propose Ivy Explainable
Detector (IVY-XDETECTOR), a unified AIGC detection and explainable architecture
that jointly performs explainable detection for both image and video content.
Our unified vision-language model achieves state-of-the-art performance across
multiple image and video detection benchmarks, highlighting the significant
advancements enabled by our dataset and modeling framework. Our data is
publicly available at https://huggingface.co/datasets/AI-Safeguard/Ivy-Fake.
How Programming Concepts and Neurons Are Shared in Code Language Models
Authors: name: Amir Hossein Kargaran, name: Yihong Liu, name: François Yvon, name: Hinrich Schütze
Published: 2025-06-01 16:24:13

Several studies have explored the mechanisms of large language models (LLMs)
in coding tasks, but most have focused on programming languages (PLs) in a
monolingual setting. In this paper, we investigate the relationship between
multiple PLs and English in the concept space of LLMs. We perform a few-shot
translation task on 21 PL pairs using two Llama-based models. By decoding the
embeddings of intermediate layers during this task, we observe that the concept
space is closer to English (including PL keywords) and assigns high
probabilities to English tokens in the second half of the intermediate layers.
We analyze neuron activations for 11 PLs and English, finding that while
language-specific neurons are primarily concentrated in the bottom layers,
those exclusive to each PL tend to appear in the top layers. For PLs that are
highly aligned with multiple other PLs, identifying language-specific neurons
is not feasible. These PLs also tend to have a larger keyword set than other
PLs and are closer to the model's concept space regardless of the input/output
PL in the translation task. Our findings provide insights into how LLMs
internally represent PLs, revealing structural patterns in the model's concept
space. Code is available at https://github.com/cisnlp/code-specific-neurons.
BinauralFlow: A Causal and Streamable Approach for High-Quality Binaural
  Speech Synthesis with Flow Matching Models
Authors: name: Susan Liang, name: Dejan Markovic, name: Israel D. Gebru, name: Steven Krenn, name: Todd Keebler, name: Jacob Sandakly, name: Frank Yu, name: Samuel Hassel, name: Chenliang Xu, name: Alexander Richard
Published: 2025-05-28 20:59:15

Binaural rendering aims to synthesize binaural audio that mimics natural
hearing based on a mono audio and the locations of the speaker and listener.
Although many methods have been proposed to solve this problem, they struggle
with rendering quality and streamable inference. Synthesizing high-quality
binaural audio that is indistinguishable from real-world recordings requires
precise modeling of binaural cues, room reverb, and ambient sounds.
Additionally, real-world applications demand streaming inference. To address
these challenges, we propose a flow matching based streaming binaural speech
synthesis framework called BinauralFlow. We consider binaural rendering to be a
generation problem rather than a regression problem and design a conditional
flow matching model to render high-quality audio. Moreover, we design a causal
U-Net architecture that estimates the current audio frame solely based on past
information to tailor generative models for streaming inference. Finally, we
introduce a continuous inference pipeline incorporating streaming STFT/ISTFT
operations, a buffer bank, a midpoint solver, and an early skip schedule to
improve rendering continuity and speed. Quantitative and qualitative
evaluations demonstrate the superiority of our method over SOTA approaches. A
perceptual study further reveals that our model is nearly indistinguishable
from real-world recordings, with a 42% confusion rate.
Neuro2Semantic: A Transfer Learning Framework for Semantic
  Reconstruction of Continuous Language from Human Intracranial EEG
Authors: name: Siavash Shams, name: Richard Antonello, name: Gavin Mischler, name: Stephan Bickel, name: Ashesh Mehta, name: Nima Mesgarani
Published: 2025-05-31 04:17:19

Decoding continuous language from neural signals remains a significant
challenge in the intersection of neuroscience and artificial intelligence. We
introduce Neuro2Semantic, a novel framework that reconstructs the semantic
content of perceived speech from intracranial EEG (iEEG) recordings. Our
approach consists of two phases: first, an LSTM-based adapter aligns neural
signals with pre-trained text embeddings; second, a corrector module generates
continuous, natural text directly from these aligned embeddings. This flexible
method overcomes the limitations of previous decoding approaches and enables
unconstrained text generation. Neuro2Semantic achieves strong performance with
as little as 30 minutes of neural data, outperforming a recent state-of-the-art
method in low-data settings. These results highlight the potential for
practical applications in brain-computer interfaces and neural decoding
technologies.
R1-Code-Interpreter: Training LLMs to Reason with Code via Supervised
  and Reinforcement Learning
Authors: name: Yongchao Chen, name: Yueying Liu, name: Junwei Zhou, name: Yilun Hao, name: Jingquan Wang, name: Yang Zhang, name: Chuchu Fan
Published: 2025-05-27 18:47:33

Despite advances in reasoning and planning of R1-like models, Large Language
Models (LLMs) still struggle with tasks requiring precise computation, symbolic
manipulation, optimization, and algorithmic reasoning, in which textual
reasoning lacks the rigor of code execution. A key challenge is enabling LLMs
to decide when to use textual reasoning versus code generation. While OpenAI
trains models to invoke a Code Interpreter as needed, public research lacks
guidance on aligning pre-trained LLMs to effectively leverage code and
generalize across diverse tasks. We present R1-Code-Interpreter, an extension
of a text-only LLM trained via multi-turn supervised fine-tuning (SFT) and
reinforcement learning (RL) to autonomously generate multiple code queries
during step-by-step reasoning. We curate 144 reasoning and planning tasks (107
for training, 37 for testing), each with over 200 diverse questions. We
fine-tune Qwen-2.5 models (3B/7B/14B) using various SFT and RL strategies,
investigating different answer formats, reasoning vs. non-reasoning models,
cold vs. warm starts, GRPO vs. PPO, and masked vs. unmasked code outputs.
Unlike prior RL work on narrow domains, we find that Code Interpreter training
is significantly harder due to high task diversity and expensive code
execution, highlighting the critical role of the SFT stage. Our final model,
R1-CI-14B, improves average accuracy on the 37 test tasks from 44.0\% to
64.1\%, outperforming GPT-4o (text-only: 58.6\%) and approaching GPT-4o with
Code Interpreter (70.9\%), with the emergent self-checking behavior via code
generation. Datasets, Codes, and Models are available at
https://github.com/yongchao98/R1-Code-Interpreter and
https://huggingface.co/yongchao98.
Frankentext: Stitching random text fragments into long-form narratives
Authors: name: Chau Minh Pham, name: Jenna Russell, name: Dzung Pham, name: Mohit Iyyer
Published: 2025-05-23 17:38:47

We introduce Frankentexts, a new type of long-form narratives produced by
LLMs under the extreme constraint that most tokens (e.g., 90%) must be copied
verbatim from human writings. This task presents a challenging test of
controllable generation, requiring models to satisfy a writing prompt,
integrate disparate text fragments, and still produce a coherent narrative. To
generate Frankentexts, we instruct the model to produce a draft by selecting
and combining human-written passages, then iteratively revise the draft while
maintaining a user-specified copy ratio. We evaluate the resulting Frankentexts
along three axes: writing quality, instruction adherence, and detectability.
Gemini-2.5-Pro performs surprisingly well on this task: 81% of its Frankentexts
are coherent and 100% relevant to the prompt. Notably, up to 59% of these
outputs are misclassified as human-written by detectors like Pangram, revealing
limitations in AI text detectors. Human annotators can sometimes identify
Frankentexts through their abrupt tone shifts and inconsistent grammar between
segments, especially in longer generations. Beyond presenting a challenging
generation task, Frankentexts invite discussion on building effective detectors
for this new grey zone of authorship, provide training data for mixed
authorship detection, and serve as a sandbox for studying human-AI co-writing
processes.
Esoteric Language Models
Authors: name: Subham Sekhar Sahoo, name: Zhihan Yang, name: Yash Akhauri, name: Johnna Liu, name: Deepansha Singh, name: Zhoujun Cheng, name: Zhengzhong Liu, name: Eric Xing, name: John Thickstun, name: Arash Vahdat
Published: 2025-06-02 17:47:27

Diffusion-based language models offer a compelling alternative to
autoregressive (AR) models by enabling parallel and controllable generation.
Among this family of models, Masked Diffusion Models (MDMs) achieve the
strongest performance but still underperform AR models in perplexity and lack
key inference-time efficiency features--most notably, KV caching. In this work,
we introduce Eso-LMs, a new family of models that fuses AR and MDM paradigms,
enabling smooth interpolation between their perplexities while overcoming their
respective limitations. Eso-LMs set a new state of the art on standard language
modeling benchmarks. Crucially, we are the **first to introduce KV caching for
MDMs** while preserving parallel generation, significantly improving inference
efficiency. Combined with an optimized sampling schedule, our method achieves
up to **65x** faster inference than standard MDMs and **4x** faster inference
than prior semi-autoregressive approaches. We provide the code and model
checkpoints on the project page:
[http://s-sahoo.github.io/Eso-LMs](http://s-sahoo.github.io/Eso-LMs)
SenseFlow: Scaling Distribution Matching for Flow-based Text-to-Image
  Distillation
Authors: name: Xingtong Ge, name: Xin Zhang, name: Tongda Xu, name: Yi Zhang, name: Xinjie Zhang, name: Yan Wang, name: Jun Zhang
Published: 2025-05-31 11:59:02

The Distribution Matching Distillation (DMD) has been successfully applied to
text-to-image diffusion models such as Stable Diffusion (SD) 1.5. However,
vanilla DMD suffers from convergence difficulties on large-scale flow-based
text-to-image models, such as SD 3.5 and FLUX. In this paper, we first analyze
the issues when applying vanilla DMD on large-scale models. Then, to overcome
the scalability challenge, we propose implicit distribution alignment (IDA) to
regularize the distance between the generator and fake distribution.
Furthermore, we propose intra-segment guidance (ISG) to relocate the timestep
importance distribution from the teacher model. With IDA alone, DMD converges
for SD 3.5; employing both IDA and ISG, DMD converges for SD 3.5 and FLUX.1
dev. Along with other improvements such as scaled up discriminator models, our
final model, dubbed SenseFlow, achieves superior performance in
distillation for both diffusion based text-to-image models such as SDXL, and
flow-matching models such as SD 3.5 Large and FLUX. The source code will be
avaliable at https://github.com/XingtongGe/SenseFlow.
Aligning VLM Assistants with Personalized Situated Cognition
Authors: name: Yongqi Li, name: Shen Zhou, name: Xiaohu Li, name: Xin Miao, name: Jintao Wen, name: Mayi Xu, name: Jianhao Chen, name: Birong Pan, name: Hankun Kang, name: Yuanyuan Zhu, name: Ming Zhong, name: Tieyun Qian
Published: 2025-06-01 09:50:54

Vision-language models (VLMs) aligned with general human objectives, such as
being harmless and hallucination-free, have become valuable assistants of
humans in managing visual tasks. However, people with diversified backgrounds
have different cognition even in the same situation. Consequently, they may
have personalized expectations for VLM assistants. This highlights the urgent
need to align VLM assistants with personalized situated cognition for
real-world assistance. To study this problem, we first simplify it by
characterizing individuals based on the sociological concept of Role-Set. Then,
we propose to evaluate the individuals' actions to examine whether the
personalized alignment is achieved. Further, we construct a benchmark named
PCogAlignBench, which includes 18k instances and 20 individuals with different
Role-Sets. Finally, we present a framework called PCogAlign, which constructs a
cognition-aware and action-based reward model for personalized alignment.
Experimental results and human evaluations demonstrate the reliability of the
PCogAlignBench and the effectiveness of our proposed PCogAlign. We will
open-source the constructed benchmark and code at
https://github.com/NLPGM/PCogAlign.
MaskSearch: A Universal Pre-Training Framework to Enhance Agentic Search
  Capability
Authors: name: Weiqi Wu, name: Xin Guan, name: Shen Huang, name: Yong Jiang, name: Pengjun Xie, name: Fei Huang, name: Jiuxin Cao, name: Hai Zhao, name: Jingren Zhou
Published: 2025-05-26 17:58:50

Retrieval-Augmented Language Models (RALMs) represent a classic paradigm
where models enhance generative capabilities using external knowledge retrieved
via a specialized module. Recent advancements in Agent techniques enable Large
Language Models (LLMs) to autonomously utilize tools for retrieval, planning,
and reasoning. While existing training-based methods show promise, their
agentic abilities are limited by inherent characteristics of the task-specific
data used during training. To further enhance the universal search capability
of agents, we propose a novel pre-training framework, MaskSearch. In the
pre-training stage, we introduce the Retrieval Augmented Mask Prediction (RAMP)
task, where the model learns to leverage search tools to fill masked spans on a
large number of pre-training data, thus acquiring universal retrieval and
reasoning capabilities for LLMs. After that, the model is trained on downstream
tasks to achieve further improvement. We apply both Supervised Fine-tuning
(SFT) and Reinforcement Learning (RL) for training. For SFT, we combine
agent-based and distillation-based methods to generate training data, starting
with a multi-agent system consisting of a planner, rewriter, observer, and
followed by a self-evolving teacher model. While for RL, we employ DAPO as the
training framework and adopt a hybrid reward system consisting of answer
rewards and format rewards. Additionally, we introduce a curriculum learning
approach that allows the model to learn progressively from easier to more
challenging instances based on the number of masked spans. We evaluate the
effectiveness of our framework in the scenario of open-domain multi-hop
question answering. Through extensive experiments, we demonstrate that
MaskSearch significantly enhances the performance of LLM-based search agents on
both in-domain and out-of-domain downstream tasks.