Orionnnn/hub-stats huggingface dataset:
How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on
  Continual Pre-Training
Authors: name: Yixin Ou, name: Yunzhi Yao, name: Ningyu Zhang, name: Hui Jin, name: Jiacheng Sun, name: Shumin Deng, name: Zhenguo Li, name: Huajun Chen
Published: 2025-02-16 16:55:43

Despite exceptional capabilities in knowledge-intensive tasks, Large Language
Models (LLMs) face a critical gap in understanding how they internalize new
knowledge, particularly how to structurally embed acquired knowledge in their
neural computations. We address this issue through the lens of knowledge
circuit evolution, identifying computational subgraphs that facilitate
knowledge storage and processing. Our systematic analysis of circuit evolution
throughout continual pre-training reveals several key findings: (1) the
acquisition of new knowledge is influenced by its relevance to pre-existing
knowledge; (2) the evolution of knowledge circuits exhibits a distinct phase
shift from formation to optimization; (3) the evolution of knowledge circuits
follows a deep-to-shallow pattern. These insights not only advance our
theoretical understanding of the mechanisms of new knowledge acquisition in
LLMs, but also provide potential implications for improving continual
pre-training strategies to enhance model performance. Code and data will be
available at https://github.com/zjunlp/DynamicKnowledgeCircuits.
SURGE: On the Potential of Large Language Models as General-Purpose
  Surrogate Code Executors
Authors: name: Bohan Lyu, name: Siqiao Huang, name: Zichen Liang
Published: 2025-02-16 15:38:19

Large language models (LLMs) have demonstrated remarkable capabilities in
code-related tasks, such as code understanding and code generation. However, an
equally important yet underexplored question is whether LLMs can serve as
general-purpose surrogate code executors, to predict the output and behavior of
a program without actually running it. To systematically investigate this
capability, we introduce SURGE, a comprehensive benchmark covering eight key
aspects: multi-language programming tasks, competition-level programming
problems, repository-level code analysis, high-cost scientific computing,
time-complexity-intensive algorithms, buggy code analysis, programs dependent
on specific compilers or execution environments, and formal mathematical proof
verification. We evaluate multiple open-source and proprietary LLMs on SURGE
and conduct a scaling study to analyze the impact of model size and training
data scale on surrogate execution accuracy. Additionally, we categorize model
prediction errors and explore potential areas for improvement. Our findings
indicate that while LLMs can predict code execution results in certain cases,
they exhibit limitations in general-purpose surrogate execution. This study
provides empirical insights into the feasibility of using LLMs as surrogate
code executors. Code and dataset are released at
https://github.com/Imbernoulli/SURGE.
ReLearn: Unlearning via Learning for Large Language Models
Authors: name: Haoming Xu, name: Ningyuan Zhao, name: Liming Yang, name: Sendong Zhao, name: Shumin Deng, name: Mengru Wang, name: Bryan Hooi, name: Nay Oo, name: Huajun Chen, name: Ningyu Zhang
Published: 2025-02-16 16:31:00

Current unlearning methods for large language models usually rely on reverse
optimization to reduce target token probabilities. However, this paradigm
disrupts the subsequent tokens prediction, degrading model performance and
linguistic coherence. Moreover, existing evaluation metrics overemphasize
contextual forgetting while inadequately assessing response fluency and
relevance. To address these challenges, we propose ReLearn, a data augmentation
and fine-tuning pipeline for effective unlearning, along with a comprehensive
evaluation framework. This framework introduces Knowledge Forgetting Rate (KFR)
and Knowledge Retention Rate (KRR) to measure knowledge-level preservation, and
Linguistic Score (LS) to evaluate generation quality. Our experiments show that
ReLearn successfully achieves targeted forgetting while preserving high-quality
output. Through mechanistic analysis, we further demonstrate how reverse
optimization disrupts coherent text generation, while ReLearn preserves this
essential capability. Code is available at https://github.com/zjunlp/unlearn.
Learning Getting-Up Policies for Real-World Humanoid Robots
Authors: name: Xialin He, name: Runpei Dong, name: Zixuan Chen, name: Saurabh Gupta
Published: 2025-02-17 18:59:06

Automatic fall recovery is a crucial prerequisite before humanoid robots can
be reliably deployed. Hand-designing controllers for getting up is difficult
because of the varied configurations a humanoid can end up in after a fall and
the challenging terrains humanoid robots are expected to operate on. This paper
develops a learning framework to produce controllers that enable humanoid
robots to get up from varying configurations on varying terrains. Unlike
previous successful applications of humanoid locomotion learning, the
getting-up task involves complex contact patterns, which necessitates
accurately modeling the collision geometry and sparser rewards. We address
these challenges through a two-phase approach that follows a curriculum. The
first stage focuses on discovering a good getting-up trajectory under minimal
constraints on smoothness or speed / torque limits. The second stage then
refines the discovered motions into deployable (i.e. smooth and slow) motions
that are robust to variations in initial configuration and terrains. We find
these innovations enable a real-world G1 humanoid robot to get up from two main
situations that we considered: a) lying face up and b) lying face down, both
tested on flat, deformable, slippery surfaces and slopes (e.g., sloppy grass
and snowfield). To the best of our knowledge, this is the first successful
demonstration of learned getting-up policies for human-sized humanoid robots in
the real world. Project page: https://humanoid-getup.github.io/
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance
  Software Engineering?
Authors: name: Samuel Miserendino, name: Michele Wang, name: Tejal Patwardhan, name: Johannes Heidecke
Published: 2025-02-17 18:41:16

We introduce SWE-Lancer, a benchmark of over 1,400 freelance software
engineering tasks from Upwork, valued at \1 million USD total in real-world
payouts. SWE-Lancer encompasses both independent engineering tasks--ranging
from 50 bug fixes to \$32,000 feature implementations--and managerial tasks,
where models choose between technical implementation proposals. Independent
tasks are graded with end-to-end tests triple-verified by experienced software
engineers, while managerial decisions are assessed against the choices of the
original hired engineering managers. We evaluate model performance and find
that frontier models are still unable to solve the majority of tasks. To
facilitate future research, we open-source a unified Docker image and a public
evaluation split, SWE-Lancer Diamond
(https://github.com/openai/SWELancer-Benchmark). By mapping model performance
to monetary value, we hope SWE-Lancer enables greater research into the
economic impact of AI model development.
video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
Authors: name: Guangzhi Sun, name: Yudong Yang, name: Jimin Zhuang, name: Changli Tang, name: Yixuan Li, name: Wei Li, name: Zejun MA, name: Chao Zhang
Published: 2025-02-17 13:07:40

While recent advancements in reasoning optimization have significantly
enhanced the capabilities of large language models (LLMs), existing efforts to
improve reasoning have been limited to solving mathematical problems and
focusing on visual graphical inputs, neglecting broader applications in general
video understanding.This paper proposes video-SALMONN-o1, the first open-source
reasoning-enhanced audio-visual LLM designed for general video understanding
tasks. To enhance its reasoning abilities, we develop a reasoning-intensive
dataset featuring challenging audio-visual questions with step-by-step
solutions. We also propose process direct preference optimization (pDPO), which
leverages contrastive step selection to achieve efficient step-level reward
modelling tailored for multimodal inputs. Additionally, we introduce RivaBench,
the first reasoning-intensive video understanding benchmark, featuring over
4,000 high-quality, expert-curated question-answer pairs across scenarios such
as standup comedy, academic presentations, and synthetic video detection.
video-SALMONN-o1 achieves 3-8% accuracy improvements over the LLaVA-OneVision
baseline across different video reasoning benchmarks. Besides, pDPO achieves
6-8% improvements compared to the supervised fine-tuning model on RivaBench.
Enhanced reasoning enables video-SALMONN-o1 zero-shot synthetic video detection
capabilities.
Talk Structurally, Act Hierarchically: A Collaborative Framework for LLM
  Multi-Agent Systems
Authors: name: Zhao Wang, name: Sota Moriyama, name: Wei-Yao Wang, name: Briti Gangopadhyay, name: Shingo Takamatsu
Published: 2025-02-16 12:26:58

Recent advancements in LLM-based multi-agent (LLM-MA) systems have shown
promise, yet significant challenges remain in managing communication and
refinement when agents collaborate on complex tasks. In this paper, we propose
Talk Structurally, Act Hierarchically (TalkHier), a novel framework
that introduces a structured communication protocol for context-rich exchanges
and a hierarchical refinement system to address issues such as incorrect
outputs, falsehoods, and biases. TalkHier surpasses various types of
SoTA, including inference scaling model (OpenAI-o1), open-source multi-agent
models (e.g., AgentVerse), and majority voting strategies on current LLM and
single-agent baselines (e.g., ReAct, GPT4o), across diverse tasks, including
open-domain question answering, domain-specific selective questioning, and
practical advertisement text generation. These results highlight its potential
to set a new standard for LLM-MA systems, paving the way for more effective,
adaptable, and collaborative multi-agent frameworks. The code is available
https://github.com/sony/talkhier.
One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual
  Reasoning in Mathematical LLMs
Authors: name: Yinghui Li, name: Jiayi Kuang, name: Haojing Huang, name: Zhikun Xu, name: Xinnian Liang, name: Yi Yu, name: Wenlian Lu, name: Yangning Li, name: Xiaoyu Tan, name: Chao Qu, name: Ying Shen, name: Hai-Tao Zheng, name: Philip S. Yu
Published: 2025-02-12 02:01:10

Leveraging mathematical Large Language Models (LLMs) for proof generation is
a fundamental topic in LLMs research. We argue that the ability of current LLMs
to prove statements largely depends on whether they have encountered the
relevant proof process during training. This reliance limits their deeper
understanding of mathematical theorems and related concepts. Inspired by the
pedagogical method of "proof by counterexamples" commonly used in human
mathematics education, our work aims to enhance LLMs' ability to conduct
mathematical reasoning and proof through counterexamples. Specifically, we
manually create a high-quality, university-level mathematical benchmark,
CounterMATH, which requires LLMs to prove mathematical statements by providing
counterexamples, thereby assessing their grasp of mathematical concepts.
Additionally, we develop a data engineering framework to automatically obtain
training data for further model improvement. Extensive experiments and detailed
analyses demonstrate that CounterMATH is challenging, indicating that LLMs,
such as OpenAI o1, have insufficient counterexample-driven proof capabilities.
Moreover, our exploration into model training reveals that strengthening LLMs'
counterexample-driven conceptual reasoning abilities is crucial for improving
their overall mathematical capabilities. We believe that our work offers new
perspectives on the community of mathematical LLMs.
Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising
  Trajectory Sharpening
Authors: name: Ye Tian, name: Ling Yang, name: Xinchen Zhang, name: Yunhai Tong, name: Mengdi Wang, name: Bin Cui
Published: 2025-02-17 18:57:26

We propose Diffusion-Sharpening, a fine-tuning approach that enhances
downstream alignment by optimizing sampling trajectories. Existing RL-based
fine-tuning methods focus on single training timesteps and neglect
trajectory-level alignment, while recent sampling trajectory optimization
methods incur significant inference NFE costs. Diffusion-Sharpening overcomes
this by using a path integral framework to select optimal trajectories during
training, leveraging reward feedback, and amortizing inference costs. Our
method demonstrates superior training efficiency with faster convergence, and
best inference efficiency without requiring additional NFEs. Extensive
experiments show that Diffusion-Sharpening outperforms RL-based fine-tuning
methods (e.g., Diffusion-DPO) and sampling trajectory optimization methods
(e.g., Inference Scaling) across diverse metrics including text alignment,
compositional capabilities, and human preferences, offering a scalable and
efficient solution for future diffusion model fine-tuning. Code:
https://github.com/Gen-Verse/Diffusion-Sharpening
HermesFlow: Seamlessly Closing the Gap in Multimodal Understanding and
  Generation
Authors: name: Ling Yang, name: Xinchen Zhang, name: Ye Tian, name: Chenming Shang, name: Minghao Xu, name: Wentao Zhang, name: Bin Cui
Published: 2025-02-17 18:57:51

The remarkable success of the autoregressive paradigm has made significant
advancement in Multimodal Large Language Models (MLLMs), with powerful models
like Show-o, Transfusion and Emu3 achieving notable progress in unified image
understanding and generation. For the first time, we uncover a common
phenomenon: the understanding capabilities of MLLMs are typically stronger than
their generative capabilities, with a significant gap between the two. Building
on this insight, we propose HermesFlow, a simple yet general framework designed
to seamlessly bridge the gap between understanding and generation in MLLMs.
Specifically, we take the homologous data as input to curate homologous
preference data of both understanding and generation. Through Pair-DPO and
self-play iterative optimization, HermesFlow effectively aligns multimodal
understanding and generation using homologous preference data. Extensive
experiments demonstrate the significant superiority of our approach over prior
methods, particularly in narrowing the gap between multimodal understanding and
generation. These findings highlight the potential of HermesFlow as a general
alignment framework for next-generation multimodal foundation models. Code:
https://github.com/Gen-Verse/HermesFlow
SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example
  Selection for Text-to-SQL
Authors: name: Jimin Lee, name: Ingeol Baek, name: Byeongjeong Kim, name: Hwanhee Lee
Published: 2025-02-17 04:52:24

Text-to-SQL aims to convert natural language questions into executable SQL
queries. While previous approaches, such as skeleton-masked selection, have
demonstrated strong performance by retrieving similar training examples to
guide large language models (LLMs), they struggle in real-world scenarios where
such examples are unavailable. To overcome this limitation, we propose
Self-Augmentation in-context learning with Fine-grained Example selection for
Text-to-SQL (SAFE-SQL), a novel framework that improves SQL generation by
generating and filtering self-augmented examples. SAFE-SQL first prompts an LLM
to generate multiple Text-to-SQL examples relevant to the test input. Then
SAFE-SQL filters these examples through three relevance assessments,
constructing high-quality in-context learning examples. Using self-generated
examples, SAFE-SQL surpasses the previous zero-shot, and few-shot Text-to-SQL
frameworks, achieving higher execution accuracy. Notably, our approach provides
additional performance gains in extra hard and unseen scenarios, where
conventional methods often fail.
CRANE: Reasoning with constrained LLM generation
Authors: name: Debangshu Banerjee, name: Tarun Suresh, name: Shubham Ugare, name: Sasa Misailovic, name: Gagandeep Singh
Published: 2025-02-13 08:23:42

Code generation, symbolic math reasoning, and other tasks require LLMs to
produce outputs that are both syntactically and semantically correct.
Constrained LLM generation is a promising direction to enforce adherence to
formal grammar, but prior works have empirically observed that strict
enforcement of formal constraints often diminishes the reasoning capabilities
of LLMs. In this work, we first provide a theoretical explanation for why
constraining LLM outputs to very restrictive grammars that only allow
syntactically valid final answers reduces the reasoning capabilities of the
model. Second, we demonstrate that by augmenting the output grammar with
carefully designed additional rules, it is always possible to preserve the
reasoning capabilities of the LLM while ensuring syntactic and semantic
correctness in its outputs. Building on these theoretical insights, we propose
a reasoning-augmented constrained decoding algorithm, CRANE, which effectively
balances the correctness of constrained generation with the flexibility of
unconstrained generation. Experiments on multiple open-source LLMs and
benchmarks show that CRANE significantly outperforms both state-of-the-art
constrained decoding strategies and standard unconstrained decoding, showing up
to 10% points accuracy improvement over baselines on challenging symbolic
reasoning benchmarks GSM-symbolic and FOLIO.
Cuckoo: An IE Free Rider Hatched by Massive Nutrition in LLM's Nest
Authors: name: Letian Peng, name: Zilong Wang, name: Feng Yao, name: Jingbo Shang
Published: 2025-02-16 21:32:20

Massive high-quality data, both pre-training raw texts and post-training
annotations, have been carefully prepared to incubate advanced large language
models (LLMs). In contrast, for information extraction (IE), pre-training data,
such as BIO-tagged sequences, are hard to scale up. We show that IE models can
act as free riders on LLM resources by reframing next-token prediction
into extraction for tokens already present in the context. Specifically,
our proposed next tokens extraction (NTE) paradigm learns a versatile IE model,
Cuckoo, with 102.6M extractive data converted from LLM's pre-training
and post-training data. Under the few-shot setting, Cuckoo adapts effectively
to traditional and complex instruction-following IE with better performance
than existing pre-trained IE models. As a free rider, Cuckoo can naturally
evolve with the ongoing advancements in LLM data preparation, benefiting from
improvements in LLM training pipelines without additional manual effort.
Building A Proof-Oriented Programmer That Is 64% Better Than GPT-4o
  Under Data Scarsity
Authors: name: Dylan Zhang, name: Justin Wang, name: Tianran Sun
Published: 2025-02-17 15:24:11

Existing LMs struggle with proof-oriented programming due to data scarcity,
which manifest in two key ways: (1) a lack of sufficient corpora for
proof-oriented programming languages such as F*, and (2) the absence of
large-scale, project-level proof-oriented implementations that can teach the
model the intricate reasoning process when performing proof-oriented
programming. We present the first on synthetic data augmentation for project
level proof oriented programming for both generation and repair. Our method
addresses data scarcity by synthesizing basic proof-oriented programming
problems for proficiency in that language; incorporating diverse coding data
for reasoning capability elicitation and creating new proofs and repair data
within existing repositories. This approach enables language models to both
synthesize and repair proofs for function- and repository-level code. We show
that our fine-tuned 14B parameter model, PoPilot, can exceed the performance of
the models that outperforms GPT-4o in project-level proof-oriented programming
by 64% relative margin, and can improve GPT-4o's performance by 54% by
repairing its outputs over GPT-4o's self-repair.
The Danger of Overthinking: Examining the Reasoning-Action Dilemma in
  Agentic Tasks
Authors: name: Alejandro Cuadron, name: Dacheng Li, name: Wenjie Ma, name: Xingyao Wang, name: Yichuan Wang, name: Siyuan Zhuang, name: Shu Liu, name: Luis Gaspar Schroeder, name: Tian Xia, name: Huanzhi Mao, name: Nicholas Thumiger, name: Aditya Desai, name: Ion Stoica, name: Ana Klimovic, name: Graham Neubig, name: Joseph E. Gonzalez
Published: 2025-02-12 09:23:26

Large Reasoning Models (LRMs) represent a breakthrough in AI problem-solving
capabilities, but their effectiveness in interactive environments can be
limited. This paper introduces and analyzes overthinking in LRMs. A phenomenon
where models favor extended internal reasoning chains over environmental
interaction. Through experiments on software engineering tasks using SWE Bench
Verified, we observe three recurring patterns: Analysis Paralysis, Rogue
Actions, and Premature Disengagement. We propose a framework to study these
behaviors, which correlates with human expert assessments, and analyze 4018
trajectories. We observe that higher overthinking scores correlate with
decreased performance, with reasoning models exhibiting stronger tendencies
toward overthinking compared to non-reasoning models. Our analysis reveals that
simple efforts to mitigate overthinking in agentic environments, such as
selecting the solution with the lower overthinking score, can improve model
performance by almost 30% while reducing computational costs by 43%. These
results suggest that mitigating overthinking has strong practical implications.
We suggest that by leveraging native function-calling capabilities and
selective reinforcement learning overthinking tendencies could be mitigated. We
also open-source our evaluation framework and dataset to facilitate research in
this direction at https://github.com/AlexCuadron/Overthinking.
Selective Self-to-Supervised Fine-Tuning for Generalization in Large
  Language Models
Authors: name: Sonam Gupta, name: Yatin Nandwani, name: Asaf Yehudai, name: Dinesh Khandelwal, name: Dinesh Raghu, name: Sachindra Joshi
Published: 2025-02-12 05:24:21

Fine-tuning Large Language Models (LLMs) on specific datasets is a common
practice to improve performance on target tasks. However, this performance gain
often leads to overfitting, where the model becomes too specialized in either
the task or the characteristics of the training data, resulting in a loss of
generalization. This paper introduces Selective Self-to-Supervised Fine-Tuning
(S3FT), a fine-tuning approach that achieves better performance than the
standard supervised fine-tuning (SFT) while improving generalization. S3FT
leverages the existence of multiple valid responses to a query. By utilizing
the model's correct responses, S3FT reduces model specialization during the
fine-tuning stage. S3FT first identifies the correct model responses from the
training set by deploying an appropriate judge. Then, it fine-tunes the model
using the correct model responses and the gold response (or its paraphrase) for
the remaining samples. The effectiveness of S3FT is demonstrated through
experiments on mathematical reasoning, Python programming and reading
comprehension tasks. The results show that standard SFT can lead to an average
performance drop of up to 4.4 on multiple benchmarks, such as MMLU and
TruthfulQA. In contrast, S3FT reduces this drop by half, i.e. 2.5, indicating
better generalization capabilities than SFT while performing significantly
better on the fine-tuning tasks.