Orionnnn/hub-stats huggingface dataset:
Architectural Backdoors for Within-Batch Data Stealing and Model
  Inference Manipulation
Authors: name: Nicolas Küchler, name: Ivan Petrov, name: Conrad Grobler, name: Ilia Shumailov
Published: 2025-05-23 19:28:45

For nearly a decade the academic community has investigated backdoors in
neural networks, primarily focusing on classification tasks where adversaries
manipulate the model prediction. While demonstrably malicious, the immediate
real-world impact of such prediction-altering attacks has remained unclear. In
this paper we introduce a novel and significantly more potent class of
backdoors that builds upon recent advancements in architectural backdoors. We
demonstrate how these backdoors can be specifically engineered to exploit
batched inference, a common technique for hardware utilization, enabling
large-scale user data manipulation and theft. By targeting the batching
process, these architectural backdoors facilitate information leakage between
concurrent user requests and allow attackers to fully control model responses
directed at other users within the same batch. In other words, an attacker who
can change the model architecture can set and steal model inputs and outputs of
other users within the same batch. We show that such attacks are not only
feasible but also alarmingly effective, can be readily injected into prevalent
model architectures, and represent a truly malicious threat to user privacy and
system integrity. Critically, to counteract this new class of vulnerabilities,
we propose a deterministic mitigation strategy that provides formal guarantees
against this new attack vector, unlike prior work that relied on Large Language
Models to find the backdoors. Our mitigation strategy employs a novel
Information Flow Control mechanism that analyzes the model graph and proves
non-interference between different user inputs within the same batch. Using our
mitigation strategy we perform a large scale analysis of models hosted through
Hugging Face and find over 200 models that introduce (unintended) information
leakage between batch entries due to the use of dynamic quantization.
Strong Membership Inference Attacks on Massive Datasets and (Moderately)
  Large Language Models
Authors: name: Jamie Hayes, name: Ilia Shumailov, name: Christopher A. Choquette-Choo, name: Matthew Jagielski, name: George Kaissis, name: Katherine Lee, name: Milad Nasr, name: Sahra Ghalebikesabi, name: Niloofar Mireshghallah, name: Meenatchi Sundaram Mutu Selva Annamalai, name: Igor Shilov, name: Matthieu Meeus, name: Yves-Alexandre de Montjoye, name: Franziska Boenisch, name: Adam Dziedzic, name: A. Feder Cooper
Published: 2025-05-24 16:23:43

State-of-the-art membership inference attacks (MIAs) typically require
training many reference models, making it difficult to scale these attacks to
large pre-trained language models (LLMs). As a result, prior research has
either relied on weaker attacks that avoid training reference models (e.g.,
fine-tuning attacks), or on stronger attacks applied to small-scale models and
datasets. However, weaker attacks have been shown to be brittle - achieving
close-to-arbitrary success - and insights from strong attacks in simplified
settings do not translate to today's LLMs. These challenges have prompted an
important question: are the limitations observed in prior work due to attack
design choices, or are MIAs fundamentally ineffective on LLMs? We address this
question by scaling LiRA - one of the strongest MIAs - to GPT-2 architectures
ranging from 10M to 1B parameters, training reference models on over 20B tokens
from the C4 dataset. Our results advance the understanding of MIAs on LLMs in
three key ways: (1) strong MIAs can succeed on pre-trained LLMs; (2) their
effectiveness, however, remains limited (e.g., AUC<0.7) in practical settings;
and, (3) the relationship between MIA success and related privacy metrics is
not as straightforward as prior work has suggested.
EquivPruner: Boosting Efficiency and Quality in LLM-Based Search via
  Action Pruning
Authors: name: Jiawei Liu, name: Qisi Chen, name: Jianshu Zhang, name: Quan Liu, name: Defu Lian
Published: 2025-05-22 07:07:43

Large Language Models (LLMs) excel at complex reasoning through search
algorithms, yet current strategies often suffer from massive token consumption
due to redundant exploration of semantically equivalent steps. Existing
semantic similarity methods struggle to accurately identify such equivalence in
domain-specific contexts like mathematical reasoning. To address this, we
propose EquivPruner, a simple yet effective approach that identifies and prunes
semantically equivalent actions during LLM reasoning search. We also introduce
MathEquiv, the first dataset we created for mathematical statement equivalence,
which enables the training of a lightweight equivalence detector. Extensive
experiments across various models and tasks demonstrate that EquivPruner
significantly reduces token consumption, improving searching efficiency and
often bolstering reasoning accuracy. For instance, when applied to
Qwen2.5-Math-7B-Instruct on GSM8K, EquivPruner reduced token consumption by
48.1\% while also improving accuracy. Our code is available at
https://github.com/Lolo1222/EquivPruner.
Surrogate Signals from Format and Length: Reinforcement Learning for
  Solving Mathematical Problems without Ground Truth Answers
Authors: name: Rihui Xin, name: Han Liu, name: Zecheng Wang, name: Yupeng Zhang, name: Dianbo Sui, name: Xiaolin Hu, name: Bingning Wang
Published: 2025-05-26 02:56:22

Large Language Models have achieved remarkable success in natural language
processing tasks, with Reinforcement Learning playing a key role in adapting
them to specific applications. However, obtaining ground truth answers for
training LLMs in mathematical problem-solving is often challenging, costly, and
sometimes unfeasible. This research delves into the utilization of format and
length as surrogate signals to train LLMs for mathematical problem-solving,
bypassing the need for traditional ground truth answers.Our study shows that a
reward function centered on format correctness alone can yield performance
improvements comparable to the standard GRPO algorithm in early phases.
Recognizing the limitations of format-only rewards in the later phases, we
incorporate length-based rewards. The resulting GRPO approach, leveraging
format-length surrogate signals, not only matches but surpasses the performance
of the standard GRPO algorithm relying on ground truth answers in certain
scenarios, achieving 40.0\% accuracy on AIME2024 with a 7B base model. Through
systematic exploration and experimentation, this research not only offers a
practical solution for training LLMs to solve mathematical problems and
reducing the dependence on extensive ground truth data collection, but also
reveals the essence of why our label-free approach succeeds: base model is like
an excellent student who has already mastered mathematical and logical
reasoning skills, but performs poorly on the test paper, it simply needs to
develop good answering habits to achieve outstanding results in exams , in
other words, to unlock the capabilities it already possesses.
The Quest for Efficient Reasoning: A Data-Centric Benchmark to CoT
  Distillation
Authors: name: Ruichen Zhang, name: Rana Muhammad Shahroz Khan, name: Zhen Tan, name: Dawei Li, name: Song Wang, name: Tianlong Chen
Published: 2025-05-24 15:54:19

Data-centric distillation, including data augmentation, selection, and
mixing, offers a promising path to creating smaller, more efficient student
Large Language Models (LLMs) that retain strong reasoning abilities. However,
there still lacks a comprehensive benchmark to systematically assess the effect
of each distillation approach. This paper introduces DC-CoT, the first
data-centric benchmark that investigates data manipulation in chain-of-thought
(CoT) distillation from method, model and data perspectives. Utilizing various
teacher models (e.g., o4-mini, Gemini-Pro, Claude-3.5) and student
architectures (e.g., 3B, 7B parameters), we rigorously evaluate the impact of
these data manipulations on student model performance across multiple reasoning
datasets, with a focus on in-distribution (IID) and out-of-distribution (OOD)
generalization, and cross-domain transfer. Our findings aim to provide
actionable insights and establish best practices for optimizing CoT
distillation through data-centric techniques, ultimately facilitating the
development of more accessible and capable reasoning models. The dataset can be
found at https://huggingface.co/datasets/rana-shahroz/DC-COT, while our code is
shared in https://anonymous.4open.science/r/DC-COT-FF4C/.
Dynamic Risk Assessments for Offensive Cybersecurity Agents
Authors: name: Boyi Wei, name: Benedikt Stroebl, name: Jiacen Xu, name: Joie Zhang, name: Zhou Li, name: Peter Henderson
Published: 2025-05-23 21:18:59

Foundation models are increasingly becoming better autonomous programmers,
raising the prospect that they could also automate dangerous offensive
cyber-operations. Current frontier model audits probe the cybersecurity risks
of such agents, but most fail to account for the degrees of freedom available
to adversaries in the real world. In particular, with strong verifiers and
financial incentives, agents for offensive cybersecurity are amenable to
iterative improvement by would-be adversaries. We argue that assessments should
take into account an expanded threat model in the context of cybersecurity,
emphasizing the varying degrees of freedom that an adversary may possess in
stateful and non-stateful environments within a fixed compute budget. We show
that even with a relatively small compute budget (8 H100 GPU Hours in our
study), adversaries can improve an agent's cybersecurity capability on
InterCode CTF by more than 40\% relative to the baseline -- without any
external assistance. These results highlight the need to evaluate agents'
cybersecurity risk in a dynamic manner, painting a more representative picture
of risk.
Enigmata: Scaling Logical Reasoning in Large Language Models with
  Synthetic Verifiable Puzzles
Authors: name: Jiangjie Chen, name: Qianyu He, name: Siyu Yuan, name: Aili Chen, name: Zhicheng Cai, name: Weinan Dai, name: Hongli Yu, name: Qiying Yu, name: Xuefeng Li, name: Jiaze Chen, name: Hao Zhou, name: Mingxuan Wang
Published: 2025-05-26 12:40:31

Large Language Models (LLMs), such as OpenAI's o1 and DeepSeek's R1, excel at
advanced reasoning tasks like math and coding via Reinforcement Learning with
Verifiable Rewards (RLVR), but still struggle with puzzles solvable by humans
without domain knowledge. We introduce Enigmata, the first comprehensive suite
tailored for improving LLMs with puzzle reasoning skills. It includes 36 tasks
across seven categories, each with 1) a generator that produces unlimited
examples with controllable difficulty and 2) a rule-based verifier for
automatic evaluation. This generator-verifier design supports scalable,
multi-task RL training, fine-grained analysis, and seamless RLVR integration.
We further propose Enigmata-Eval, a rigorous benchmark, and develop optimized
multi-task RLVR strategies. Our trained model, Qwen2.5-32B-Enigmata,
consistently surpasses o3-mini-high and o1 on the puzzle reasoning benchmarks
like Enigmata-Eval, ARC-AGI (32.8%), and ARC-AGI 2 (0.6%). It also generalizes
well to out-of-domain puzzle benchmarks and mathematical reasoning, with little
multi-tasking trade-off. When trained on larger models like Seed1.5-Thinking
(20B activated parameters and 200B total parameters), puzzle data from Enigmata
further boosts SoTA performance on advanced math and STEM reasoning tasks such
as AIME (2024-2025), BeyondAIME and GPQA (Diamond), showing nice generalization
benefits of Enigmata. This work offers a unified, controllable framework for
advancing logical reasoning in LLMs. Resources of this work can be found at
https://seed-enigmata.github.io.
Interleaved Reasoning for Large Language Models via Reinforcement
  Learning
Authors: name: Roy Xie, name: David Qiu, name: Deepak Gopinath, name: Dong Lin, name: Yanchao Sun, name: Chong Wang, name: Saloni Potdar, name: Bhuwan Dhingra
Published: 2025-05-26 07:58:17

Long chain-of-thought (CoT) significantly enhances large language models'
(LLM) reasoning capabilities. However, the extensive reasoning traces lead to
inefficiencies and an increased time-to-first-token (TTFT). We propose a novel
training paradigm that uses reinforcement learning (RL) to guide reasoning LLMs
to interleave thinking and answering for multi-hop questions. We observe that
models inherently possess the ability to perform interleaved reasoning, which
can be further enhanced through RL. We introduce a simple yet effective
rule-based reward to incentivize correct intermediate steps, which guides the
policy model toward correct reasoning paths by leveraging intermediate signals
generated during interleaved reasoning. Extensive experiments conducted across
five diverse datasets and three RL algorithms (PPO, GRPO, and REINFORCE++)
demonstrate consistent improvements over traditional think-answer reasoning,
without requiring external tools. Specifically, our approach reduces TTFT by
over 80% on average and improves up to 19.3% in Pass@1 accuracy. Furthermore,
our method, trained solely on question answering and logical reasoning
datasets, exhibits strong generalization ability to complex reasoning datasets
such as MATH, GPQA, and MMLU. Additionally, we conduct in-depth analysis to
reveal several valuable insights into conditional reward modeling.
Hard Negative Contrastive Learning for Fine-Grained Geometric
  Understanding in Large Multimodal Models
Authors: name: Kai Sun, name: Yushi Bai, name: Zhen Yang, name: Jiajie Zhang, name: Ji Qi, name: Lei Hou, name: Juanzi Li
Published: 2025-05-26 15:55:28

Benefiting from contrastively trained visual encoders on large-scale natural
scene images, Large Multimodal Models (LMMs) have achieved remarkable
performance across various visual perception tasks. However, the inherent
limitations of contrastive learning upon summarized descriptions fundamentally
restrict the capabilities of models in meticulous reasoning, particularly in
crucial scenarios of geometric problem-solving. To enhance geometric
understanding, we propose a novel hard negative contrastive learning framework
for the vision encoder, which combines image-based contrastive learning using
generation-based hard negatives created by perturbing diagram generation code,
and text-based contrastive learning using rule-based negatives derived from
modified geometric descriptions and retrieval-based negatives selected based on
caption similarity. We train CLIP using our strong negative learning method,
namely MMCLIP (Multimodal Math CLIP), and subsequently train an LMM for
geometric problem-solving. Experiments show that our trained model, MMGeoLM,
significantly outperforms other open-source models on three geometric reasoning
benchmarks. Even with a size of 7B, it can rival powerful closed-source models
like GPT-4o. We further study the impact of different negative sample
construction methods and the number of negative samples on the geometric
reasoning performance of LMM, yielding fruitful conclusions. The code and
dataset are available at https://github.com/THU-KEG/MMGeoLM.
BizFinBench: A Business-Driven Real-World Financial Benchmark for
  Evaluating LLMs
Authors: name: Guilong Lu, name: Xuntao Guo, name: Rongjunchen Zhang, name: Wenqiao Zhu, name: Ji Liu
Published: 2025-05-26 03:23:02

Large language models excel in general tasks, yet assessing their reliability
in logic-heavy, precision-critical domains like finance, law, and healthcare
remains challenging. To address this, we introduce BizFinBench, the first
benchmark specifically designed to evaluate LLMs in real-world financial
applications. BizFinBench consists of 6,781 well-annotated queries in Chinese,
spanning five dimensions: numerical calculation, reasoning, information
extraction, prediction recognition, and knowledge-based question answering,
grouped into nine fine-grained categories. The benchmark includes both
objective and subjective metrics. We also introduce IteraJudge, a novel LLM
evaluation method that reduces bias when LLMs serve as evaluators in objective
metrics. We benchmark 25 models, including both proprietary and open-source
systems. Extensive experiments show that no model dominates across all tasks.
Our evaluation reveals distinct capability patterns: (1) In Numerical
Calculation, Claude-3.5-Sonnet (63.18) and DeepSeek-R1 (64.04) lead, while
smaller models like Qwen2.5-VL-3B (15.92) lag significantly; (2) In Reasoning,
proprietary models dominate (ChatGPT-o3: 83.58, Gemini-2.0-Flash: 81.15), with
open-source models trailing by up to 19.49 points; (3) In Information
Extraction, the performance spread is the largest, with DeepSeek-R1 scoring
71.46, while Qwen3-1.7B scores 11.23; (4) In Prediction Recognition,
performance variance is minimal, with top models scoring between 39.16 and
50.00. We find that while current LLMs handle routine finance queries
competently, they struggle with complex scenarios requiring cross-concept
reasoning. BizFinBench offers a rigorous, business-aligned benchmark for future
research. The code and dataset are available at
https://github.com/HiThink-Research/BizFinBench.
Embodied Agents Meet Personalization: Exploring Memory Utilization for
  Personalized Assistance
Authors: name: Taeyoon Kwon, name: Dongwook Choi, name: Sunghwan Kim, name: Hyojun Kim, name: Seungjun Moon, name: Beong-woo Kwak, name: Kuan-Hao Huang, name: Jinyoung Yeo
Published: 2025-05-22 08:00:10

Embodied agents empowered by large language models (LLMs) have shown strong
performance in household object rearrangement tasks. However, these tasks
primarily focus on single-turn interactions with simplified instructions, which
do not truly reflect the challenges of providing meaningful assistance to
users. To provide personalized assistance, embodied agents must understand the
unique semantics that users assign to the physical world (e.g., favorite cup,
breakfast routine) by leveraging prior interaction history to interpret
dynamic, real-world instructions. Yet, the effectiveness of embodied agents in
utilizing memory for personalized assistance remains largely underexplored. To
address this gap, we present MEMENTO, a personalized embodied agent evaluation
framework designed to comprehensively assess memory utilization capabilities to
provide personalized assistance. Our framework consists of a two-stage memory
evaluation process design that enables quantifying the impact of memory
utilization on task performance. This process enables the evaluation of agents'
understanding of personalized knowledge in object rearrangement tasks by
focusing on its role in goal interpretation: (1) the ability to identify target
objects based on personal meaning (object semantics), and (2) the ability to
infer object-location configurations from consistent user patterns, such as
routines (user patterns). Our experiments across various LLMs reveal
significant limitations in memory utilization, with even frontier models like
GPT-4o experiencing a 30.5% performance drop when required to reference
multiple memories, particularly in tasks involving user patterns. These
findings, along with our detailed analyses and case studies, provide valuable
insights for future research in developing more effective personalized embodied
agents. Project website: https://connoriginal.github.io/MEMENTO
Lifelong Safety Alignment for Language Models
Authors: name: Haoyu Wang, name: Zeyu Qin, name: Yifei Zhao, name: Chao Du, name: Min Lin, name: Xueqian Wang, name: Tianyu Pang
Published: 2025-05-26 17:40:40

LLMs have made impressive progress, but their growing capabilities also
expose them to highly flexible jailbreaking attacks designed to bypass safety
alignment. While many existing defenses focus on known types of attacks, it is
more critical to prepare LLMs for unseen attacks that may arise during
deployment. To address this, we propose a lifelong safety alignment framework
that enables LLMs to continuously adapt to new and evolving jailbreaking
strategies. Our framework introduces a competitive setup between two
components: a Meta-Attacker, trained to actively discover novel jailbreaking
strategies, and a Defender, trained to resist them. To effectively warm up the
Meta-Attacker, we first leverage the GPT-4o API to extract key insights from a
large collection of jailbreak-related research papers. Through iterative
training, the first iteration Meta-Attacker achieves a 73% attack success rate
(ASR) on RR and a 57% transfer ASR on LAT using only single-turn attacks.
Meanwhile, the Defender progressively improves its robustness and ultimately
reduces the Meta-Attacker's success rate to just 7%, enabling safer and more
reliable deployment of LLMs in open-ended environments. The code is available
at https://github.com/sail-sg/LifelongSafetyAlignment.
Shifting AI Efficiency From Model-Centric to Data-Centric Compression
Authors: name: Xuyang Liu, name: Zichen Wen, name: Shaobo Wang, name: Junjie Chen, name: Zhishan Tao, name: Yubo Wang, name: Xiangqi Jin, name: Chang Zou, name: Yiyu Wang, name: Chenfei Liao, name: Xu Zheng, name: Honggang Chen, name: Weijia Li, name: Xuming Hu, name: Conghui He, name: Linfeng Zhang
Published: 2025-05-25 13:51:17

The rapid advancement of large language models (LLMs) and multi-modal LLMs
(MLLMs) has historically relied on model-centric scaling through increasing
parameter counts from millions to hundreds of billions to drive performance
gains. However, as we approach hardware limits on model size, the dominant
computational bottleneck has fundamentally shifted to the quadratic cost of
self-attention over long token sequences, now driven by ultra-long text
contexts, high-resolution images, and extended videos. In this position paper,
we argue that the focus of research for efficient AI is shifting from
model-centric compression to data-centric compression. We position token
compression as the new frontier, which improves AI efficiency via reducing the
number of tokens during model training or inference. Through comprehensive
analysis, we first examine recent developments in long-context AI across
various domains and establish a unified mathematical framework for existing
model efficiency strategies, demonstrating why token compression represents a
crucial paradigm shift in addressing long-context overhead. Subsequently, we
systematically review the research landscape of token compression, analyzing
its fundamental benefits and identifying its compelling advantages across
diverse scenarios. Furthermore, we provide an in-depth analysis of current
challenges in token compression research and outline promising future
directions. Ultimately, our work aims to offer a fresh perspective on AI
efficiency, synthesize existing research, and catalyze innovative developments
to address the challenges that increasing context lengths pose to the AI
community's advancement.