dataset-review/dataset-review huggingface dataset:
ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning
ICML 2025

SHIELDAGENT: Shielding Agents via Verifiable Safety Policy Reasoning
Zhaorun Chen 1 Mintong Kang 2 Bo Li 1 2
Abstract
Autonomous agents powered by foundation mod-
els have seen widespread adoption across vari-
ous real-world applications. However, they re-
main highly vulnerable to malicious instructions
and attacks, which can result in severe conse-
quences such as privacy breaches and financial
losses. More critically, existing guardrails for
LLMs are not applicable due to the complex
and dynamic nature of agents. To tackle these
challenges, we propose SHIELDAGENT, the first
guardrail agent designed to enforce explicit safety
policy compliance for the action trajectory of
other protected agents through logical reason-
ing. Specifically, SHIELDAGENT first constructs
a safety policy model by extracting verifiable
rules from policy documents and structuring them
into a set of action-based probabilistic rule cir-
cuits. Given the action trajectory of the protected
agent, SHIELDAGENT retrieves relevant rule cir-
cuits and generates a shielding plan, leveraging its
comprehensive tool library and executable code
for formal verification. In addition, given the
lack of guardrail benchmarks for agents, we in-
troduce SHIELDAGENT-BENCH, a dataset with
3K safety-related pairs of agent instructions and
action trajectories, collected via SOTA attacks
across 6 web environments and 7 risk categories.
Experiments show that SHIELDAGENT achieves
SOTA on SHIELDAGENT-BENCH and three ex-
isting benchmarks, outperforming prior meth-
ods by 11.3% on average with a high recall of
90.1%.
Additionally, SHIELDAGENT reduces
API queries by 64.7% and inference time by
58.2%, demonstrating its high precision and effi-
ciency in safeguarding agents. Our project is avail-
able and continuously maintained here: https:
//shieldagent-aiguard.github.io/
1University of Chicago, Chicago IL, USA 2University of Illinois
at Urbana-Champaign, Champaign IL, USA. Correspondence to:
Zhaorun Chen, Bo Li <{zhaor...
DictPFL: Efficient and Private Federated Learning on Encrypted Gradients
NeurIPS 2025

DictPFL: Efficient and Private Federated Learning
on Encrypted Gradients
Jiaqi Xue1, Mayank Kumar1, Yuzhang Shang1, Shangqian Gao2, Rui Ning3
Mengxin Zheng1, Xiaoqian Jiang4, Qian Lou1
1University of Central Florida
2Florida State University
3Old Dominion University
4University of Texas Health Science Center at Houston
{jiaqi.xue,mayank.kumar,yuzhang.shang,mengxin.zheng,qian.lou}@ucf.edu
sgao@cs.fsu.edu, rning@odu.edu, xiaoqian.jiang@uth.tmc.edu
Abstract
Federated Learning (FL) enables collaborative model training across institutions
without sharing raw data. However, gradient sharing still risks privacy leakage,
such as gradient inversion attacks. Homomorphic Encryption (HE) can secure
aggregation but often incurs prohibitive computational and communication over-
head. Existing HE-based FL methods sit at two extremes: encrypting all gradients
for full privacy at high cost, or partially encrypting gradients to save resources
while exposing vulnerabilities. We present DictPFL, a practical framework that
achieves full gradient protection with minimal overhead. DictPFL encrypts every
transmitted gradient while keeping non-transmitted parameters local, preserving
privacy without heavy computation. It introduces two key modules: Decompose-
for-Partial-Encrypt (DePE), which decomposes model weights into a static dic-
tionary and an updatable lookup table—only the latter is encrypted and aggregated,
while the static dictionary remains local and requires neither sharing nor encryp-
tion; and Prune-for-Minimum-Encrypt (PrME), which applies encryption-aware
pruning to minimize encrypted parameters via consistent, history-guided masks.
Experiments show that DictPFL reduces communication cost by 402–748× and
accelerates training by 28–65× compared to fully encrypted FL, while outper-
forming state-of-the-art selective encryption methods by 51–155× in overhead
and 4–19× in speed. Remarkably, DictPFL’s runtime is within 2× of plaintext
FL, demonstrating—for the first time—that HE...
Fictitious Synthetic Data Can Improve LLM Factuality via Prerequisite Learning
ICLR 2025

Published as a conference paper at ICLR 2025
FICTITIOUS SYNTHETIC DATA CAN IMPROVE LLM
FACTUALITY VIA PREREQUISITE LEARNING
Yujian Liu
UC Santa Barbara
yujianliu@ucsb.edu
Shiyu Chang
UC Santa Barbara
chang87@ucsb.edu
Tommi Jaakkola
MIT CSAIL
tommi@csail.mit.edu
Yang Zhang
MIT-IBM Watson AI Lab
yang.zhang2@ibm.com
ABSTRACT
Recent studies have identified one aggravating factor of LLM hallucinations as
the knowledge inconsistency between pre-training and fine-tuning, where unfa-
miliar fine-tuning data mislead the LLM to fabricate plausible but wrong outputs.
In this paper, we propose a novel fine-tuning strategy called PREREQ-TUNE to
address this knowledge inconsistency and reduce hallucinations. Fundamentally,
PREREQ-TUNE disentangles the learning of skills and knowledge, so the model
learns only the task skills without being impacted by the knowledge inconsistency.
To achieve this, PREREQ-TUNE introduces an additional prerequisite learning
stage to learn the necessary knowledge for SFT, allowing subsequent SFT to focus
only on task skills. PREREQ-TUNE can also be combined with fictitious synthetic
data to enhance the grounding of LLM outputs to their internal knowledge. Ex-
periments show that PREREQ-TUNE outperforms existing baselines in improving
LLM’s factuality across short QA and long-form generation tasks. It also opens
new possibilities for knowledge-controlled generation in LLMs. Our code is avail-
able at https://github.com/UCSB-NLP-Chang/Prereq_tune.git.
1
INTRODUCTION
Hallucination of large language models (LLMs) refers to the phenomenon where LLMs’ outputs
look plausible but diverge from real-world facts. It has become a major concern of LLMs, seriously
undermining their reliability and trustworthiness (Huang et al., 2023; Ji et al., 2023).
Recent research has unveiled one aggravating factor of LLM hallucination, which is the knowledge
inconsistency between the pre-training and tuning (e.g., instruction- or fine-tuning) stages (Gekhman
et al., 2024; Kang...
What If We Recaption Billions of Web Images with LLaMA-3?
ICML 2025

What If We Recaption Billions of Web Images
with LLaMA-3?
Xianhang Li * 1 Haoqin Tu * 1 Mude Hui * 1 Zeyu Wang * 1 Bingchen Zhao * 2 Junfei Xiao 3 Sucheng Ren 3
Jieru Mei 3 Qing Liu 4 Huangjie Zheng 5 Yuyin Zhou 1 Cihang Xie 1
Abstract
Web-crawled image-text pairs are inherently noisy.
Prior studies demonstrate that semantically align-
ing and enriching textual descriptions of these
pairs can significantly enhance model training
across various vision-language tasks, particularly
text-to-image generation. However, large-scale
investigations in this area remain predominantly
closed-source.
Our paper aims to bridge this
community effort, leveraging the powerful and
open-sourced LLaMA-3, a GPT-4 level LLM.
Our recaptioning pipeline is simple: first, we
fine-tune a LLaMA-3-8B powered LLaVA-1.5
and then employ it to recaption ∼1.3 billion im-
ages from the DataComp-1B dataset. Our em-
pirical results confirm that this enhanced dataset,
Recap-DataComp-1B, offers substantial benefits
in training advanced vision-language models. For
discriminative models like CLIP, we observe an
average of 3.1% enhanced zero-shot performance
cross four cross-modal retrieval tasks using a
mixed set of the original and our captions. For
generative models like text-to-image Diffusion
Transformers, the generated images exhibit a sig-
nificant improvement in alignment with users’
text instructions, especially in following complex
queries.
Our project page is https://www.
haqtu.me/Recap-Datacomp-1B/.
1. Introduction
The exponential growth in data availability is one of the most
paramount factors in driving the monumental successes of
deep learning over the past decade (Deng et al., 2009; Lin
et al., 2014; Changpinyo et al., 2021; Schuhmann et al.,
*Equal contribution
1University of California, Santa Cruz
2University of Edinburgh 3Johns Hopkins University 4Adobe Inc.
5University of Texas at Austin. Correspondence to: Cihang Xie
<cixie@ucsc.edu>.
Proceedings of the 42 nd International Conference on ...
DINGO: Constrained Inference for Diffusion LLMs
NeurIPS 2025

DINGO: Constrained Inference for Diffusion LLMs
Tarun Suresh∗, Debangshu Banerjee∗, Shubham Ugare, Sasa Misailovic, Gagandeep Singh
Department of Computer Science,
University of Illinois Urbana-Champaign
Abstract
Diffusion LLMs have emerged as a promising alternative to conventional autore-
gressive LLMs, offering substantial potential for improving runtime efficiency.
However, existing diffusion models fail to provably enforce user-specified formal
constraints, such as regular expressions, which makes them unreliable for tasks
that require structured outputs, such as fixed-schema JSON generation. Unlike
autoregressive models, which generate tokens sequentially, diffusion LLMs predict
a block of tokens in parallel. This parallelism makes traditional constrained decod-
ing algorithms, designed to enforce constraints with sequential token prediction,
ineffective at preserving the true output distribution. To address this limitation, we
propose DINGO, a dynamic programming-based constrained decoding strategy
that is both efficient and provably distribution-preserving. DINGO enables sam-
pling of output strings with the highest probability under the model’s predicted
distribution while strictly adhering to any user-specified regular expression. On
standard symbolic math and JSON generation benchmarks, DINGO achieves up to
a 68% points of improvement over unconstrained inference. The code is available
at DINGO.
1
Introduction
Autoregressive LLMs demonstrate impressive performance across a wide range of tasks, including
logical reasoning [Pan et al., 2023], theorem proving [Yang et al., 2023], and code generation [et.
al., 2021]. However, because they generate one token at a time, they can be slow when producing
long responses. Recent work has explored using diffusion models to accelerate token generation by
predicting blocks of tokens in parallel. For tasks such as logical reasoning, where the LLM output
is fed into symbolic solvers like Z3 [Fedoseev et al., 2024], synta...
TRAP: Targeted Redirecting of Agentic Preferences
NeurIPS 2025

TRAP: Targeted Redirecting of Agentic Preferences
Hangoo Kang∗1, Jehyeok Yeon∗1,Gagandeep Singh1
1University of Illinois Urbana-Champaign
{hangook2,jehyeok2,ggnds}@illinois.edu
Abstract
Autonomous agentic AI systems powered by vision-language models (VLMs)
are rapidly advancing toward real-world deployment, yet their cross-modal rea-
soning capabilities introduce new attack surfaces for adversarial manipulation
that exploit semantic reasoning across modalities. Existing adversarial attacks
typically rely on visible pixel perturbations or require privileged model or en-
vironment access, making them impractical for stealthy, real-world exploitation.
We introduce TRAP, a novel generative adversarial framework that manipulates
the agent’s decision-making using diffusion-based semantic injections into the
vision-language embedding space. Our method combines negative prompt–based
degradation with positive semantic optimization, guided by a Siamese semantic
network and layout-aware spatial masking. Without requiring access to model
internals, TRAP produces visually natural images yet induces consistent selection
biases in agentic AI systems. We evaluate TRAP on the Microsoft Common
Objects in Context (COCO) dataset, building multi-candidate decision scenar-
ios. Across these scenarios, TRAP consistently induces decision-level preference
redirection on leading models, including LLaVA-34B, Gemma3, GPT-4o, and
Mistral-3.2, significantly outperforming existing baselines such as SPSA, Bandit,
and standard diffusion approaches. These findings expose a critical, generalized
vulnerability: autonomous agents can be consistently misled through visually sub-
tle, semantically-guided cross-modal manipulations. Overall, our results show
the need for defense strategies beyond pixel-level robustness to address semantic
vulnerabilities in cross-modal decision-making. The code for TRAP is accessible
on GitHub at https://github.com/uiuc-focal-lab/TRAP.
1
Introduction
Vision-Language Models...
UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction
ICML 2025

UI-Vision: A Desktop-centric GUI Benchmark for
Visual Perception and Interaction
Shravan Nayak * 1 2 3 Xiangru Jian * 4 3 Kevin Qinghong Lin 5 Juan A. Rodriguez 1 3 6 Montek Kalsi 3
Nicolas Chapados 3 M. Tamer ¨Ozsu 4 Aishwarya Agrawal 1 2 7 David Vazquez 3 Christopher Pal 1 8 3 7
Perouz Taslakian 3 Spandana Gella 3 Sai Rajeswar 3 1 2
https://uivision.github.io
Abstract
Autonomous agents that navigate Graphical User
Interfaces (GUIs) to automate tasks like docu-
ment editing and file management can greatly
enhance computer workflows. While existing re-
search focuses on online settings, desktop environ-
ments, critical for many professional and everyday
tasks, remain underexplored due to data collec-
tion challenges and licensing issues. We intro-
duce UI-Vision, the first comprehensive, license-
permissive benchmark for offline, fine-grained
evaluation of computer use agents in real-world
desktop environments. Unlike online benchmarks,
UI-Vision provides: (i) dense, high-quality an-
notations of human demonstrations, including
bounding boxes, UI labels, and action trajecto-
ries (clicks, drags, and keyboard inputs) across
83 software applications, and (ii) three fine-to-
coarse grained tasks—Element Grounding, Lay-
out Grounding, and Action Prediction—with
well-defined metrics to rigorously evaluate agents’
performance in desktop environments. Our evalu-
ation reveals critical limitations in state-of-the-art
models like UI-TARS-72B, including issues with
understanding professional software, spatial rea-
soning, and complex actions like drag-and-drop.
These findings highlight the challenges in develop-
ing fully autonomous computer-use agents. With
UI-Vision, we aim to advance the development of
more capable agents for real-world desktop tasks.
*Equal contribution
1Mila - Quebec AI Institute 2Universit´e
de Montr´eal
3ServiceNow Research
4University of Water-
loo 5National University of Singapore 6 ´Ecole de Technologie
Sup´erieure 7CIFAR AI Chair 8Polytechnique Mon...
Addressing Mark Imbalance in Integration-free Marked Temporal Point Processes
NeurIPS 2025

Addressing Mark Imbalance in Integration-free
Neural Marked Temporal Point Processes
Sishun Liu
RMIT University
Melbourne, Victoria 3000
sishun.liu@student.rmit.edu.au
Ke Deng
RMIT University
Melbourne, Victoria 3000
ke.deng@rmit.edu.au
Yongli Ren
RMIT University
Melbourne, Victoria 3000
yongli.ren@rmit.edu.au
Yan Wang
Macquarie University
Syndey, New South Wales 2000
yan.wang@mq.edu.au
Xiuzhen Zhang
RMIT University
Melbourne, Victoria 3000
xiuzhen.zhang@rmit.edu.au
Abstract
Marked Temporal Point Process (MTPP) has been well studied to model the
event distribution in marked event streams, which can be used to predict the mark
and arrival time of the next event. However, existing studies overlook that the
distribution of event marks is highly imbalanced in many real-world applications,
with some marks being frequent but others rare. The imbalance poses a significant
challenge to the performance of the next event prediction, especially for events of
rare marks. To address this issue, we propose a thresholding method, which learns
thresholds to tune the mark probability normalized by the mark’s prior probability
to optimize mark prediction, rather than predicting the mark directly based on
the mark probability as in existing studies. In conjunction with this method, we
predict the mark first and then the time. In particular, we develop a novel neural
MTPP model to support effective time sampling and estimation of mark probability
without computationally expensive numerical improper integration. Extensive
experiments on real-world datasets demonstrate the superior performance of our
solution against various baselines for the next event mark and time prediction. The
code is available at https://github.com/undes1red/IFNMTPP.
1
Introduction
Marked Temporal Point Process (MTPP) models event sequences observed from natural phenomena
(e.g. earthquakes) or generated in human activities (e.g. retweets), where each event has a mark and
an arrival time. MTPP has attracted the att...
Scaling Test-Time Compute Without Verification or RL is Suboptimal
ICML 2025

Scaling Test-Time Compute Without Verification or RL is Suboptimal
Amrith Setlur 1 Nived Rajaraman 2 Sergey Levine 2 Aviral Kumar 1
Abstract
Despite substantial advances in scaling test-time
compute, an ongoing debate in the community is
how it should be scaled up to enable continued
and efficient improvements with scaling. There
are largely two approaches: (i) distilling success-
ful search or thinking traces; and (ii), using ver-
ification (e.g., 0/1 outcome rewards, or verifiers)
to guide reinforcement learning (RL) and search
algorithms. In this paper, we prove that finetuning
LLMs with verifier-based (VB) methods based
on RL or search is far superior to verifier-free
(VF) approaches based on distilling or cloning
search traces, given a fixed amount of compute/-
data budget. Further, we show that as we scale
test-time compute (measured as the output token
length) and training data, suboptimality of VF
methods scales poorly compared to VB when the
base pre-trained LLM presents a heterogeneous
distribution over correct solution traces (e.g., dif-
ferent lengths, styles, etc.) and admits a non-
sharp distribution over rewards on traces sampled
from it. We formalize this condition using anti-
concentration (Erd¨os, 1945), implying a stronger
result that VB methods scale better asymptoti-
cally, with the performance gap between VB and
VF widening as test-time budget grows. We cor-
roborate our theory empirically on didactic and
math reasoning problems with 3/8/32B-sized pre-
trained LLMs, where we find verification is cru-
cial for scaling test-time compute.
1. Introduction
Pre-training and post-training of LLMs rely heavily on ac-
cess to high-quality “expert” data, but it is projected that by
2028, the availability of such data on the Internet will dimin-
ish (Villalobos et al., 2022; Liu et al., 2024), and improving
model performance on several domains (e.g., reasoning,
safety, etc.) often requires more data (Li et al., 2024). As a
1Carnegie Mellon University 2UC ...
ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals
ICML 2025

ResQ: Mixed-Precision Quantization of Large Language Models
with Low-Rank Residuals
Utkarsh Saxena 1 Sayeh Sharify 2 Kaushik Roy 1 Xin Wang 2
Abstract
Post-training quantization (PTQ) of large lan-
guage models (LLMs) holds the promise in re-
ducing the prohibitive computational cost at infer-
ence time. Quantization of all weight, activation
and key-value (KV) cache tensors to 4-bit without
significantly degrading generalizability is chal-
lenging, due to the high quantization error caused
by extreme outliers in activations. To tackle this
problem, we propose ResQ, a PTQ method that
pushes further the state-of-the-art. By means of
principal component analysis (PCA), it identifies
a low-rank subspace (in practice 1/8 of the hidden
dimension) in which activation variances are high-
est, and keep the coefficients within this subspace
in high precision, e.g. 8-bit, while quantizing the
rest to 4-bit. Within each subspace, invariant ran-
dom rotation is applied to further suppress outliers.
We show that this is a provably optimal mixed pre-
cision quantization scheme that minimizes error.
With the Llama and Qwen2.5 families of mod-
els, we demonstrate that ResQ outperforms recent
uniform and mixed precision PTQ methods on
a variety of benchmarks, achieving up to 33%
lower perplexity on Wikitext than the next best
method SpinQuant, and upto 5× speedup over
16-bit baseline. Code is available here.1
1. Introduction
Growing capabilities of large language models (LLMs)
come with an increasing computational cost at inference
time. LLM inference has two distinct stages: prefilling,
which processes the input prompt and populates the internal
1Department of Electrical and Computer Engineering, Purdue
University, West Lafayette, USA 2d-Matrix, Santa Clara, USA.
Correspondence to: Utkarsh Saxena <saxenau@purdue.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
1https://github.com/utkarsh...
Statistical Hypothesis Testing for Auditing Robustness in Language Models
ICML 2025

Statistical Hypothesis Testing for Auditing Robustness in Language Models
Paulius Rauba 1 Qiyao Wei 1 Mihaela van der Schaar 1
Abstract
Consider the problem of testing whether the out-
puts of a large language model (LLM) system
change under an arbitrary intervention, such as
an input perturbation or changing the model vari-
ant. We cannot simply compare two LLM out-
puts since they might differ due to the stochastic
nature of the system, nor can we compare the
entire output distribution due to computational
intractability. While existing methods for analyz-
ing text-based outputs exist, they focus on fun-
damentally different problems, such as measur-
ing bias or fairness. To this end, we introduce
distribution-based perturbation analysis, a frame-
work that reformulates LLM perturbation analysis
as a frequentist hypothesis testing problem. We
construct empirical null and alternative output dis-
tributions within a low-dimensional semantic sim-
ilarity space via Monte Carlo sampling, enabling
tractable inference without restrictive distribu-
tional assumptions. The framework is (i) model-
agnostic, (ii) supports the evaluation of arbitrary
input perturbations on any black-box LLM, (iii)
yields interpretable p-values; (iv) supports multi-
ple perturbations via controlled error rates; and (v)
provides scalar effect sizes. We demonstrate the
usefulness of the framework across multiple case
studies, showing how we can quantify response
changes, measure true/false positive rates, and
evaluate alignment with reference models. Above
all, we see this as a reliable frequentist hypothesis
testing framework for LLM auditing.
1. Introduction
Large language models (LLMs) generate outputs condi-
tioned on textual inputs by iteratively sampling from a dis-
tribution of tokens. Therefore, the outputs of LLMs exhibit
1University of Cambridge. Correspondence to: Paulius Rauba
<pr501@cam.ac.uk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada....