dataset-review/dataset-review huggingface dataset:
CurvGAD: Leveraging Curvature for Enhanced Graph Anomaly Detection
ICML 2025

CurvGAD: Leveraging Curvature for Enhanced Graph Anomaly Detection
Karish Grover 1 Geoffrey J. Gordon 1 Christos Faloutsos 1
Abstract
Does the intrinsic curvature of complex net-
works hold the key to unveiling graph anoma-
lies that conventional approaches overlook?
Reconstruction-based graph anomaly detection
(GAD) methods overlook such geometric outliers,
focusing only on structural and attribute-level
anomalies. To this end, we propose CurvGAD
— a mixed-curvature graph autoencoder that in-
troduces the notion of curvature-based geomet-
ric anomalies. CurvGAD introduces two parallel
pipelines for enhanced anomaly interpretability:
(1) Curvature-equivariant geometry reconstruc-
tion, which focuses exclusively on reconstruct-
ing the edge curvatures using a mixed-curvature,
Riemannian encoder and Gaussian kernel-based
decoder; and (2) Curvature-invariant structure
and attribute reconstruction, which decouples
structural and attribute anomalies from geomet-
ric irregularities by regularizing graph curvature
under discrete Ollivier-Ricci flow, thereby isolat-
ing the non-geometric anomalies. By leveraging
curvature, CurvGAD refines the existing anomaly
classifications and identifies new curvature-driven
anomalies. Extensive experimentation over 10
real-world datasets (both homophilic and het-
erophilic) demonstrates an improvement of up to
6.5% over state-of-the-art GAD methods. The
code is available at:
https://github.com/
karish-grover/curvgad.
1. Introduction
Detecting anomalies in graph-structured data is a pivotal
task across diverse fields, including social networks (Has-
sanzadeh et al., 2012), cybersecurity (Wang & Zhu, 2022),
transportation systems (Hu et al., 2020), and biological net-
works (Singh & Vig, 2017). Traditional methods primarily
1Carnegie Mellon University.
Correspondence to: Karish
Grover <karishg@cs.cmu.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(...
Preference Learning for AI Alignment: a Causal Perspective
ICML 2025

Preference Learning for AI Alignment: a Causal Perspective
Katarzyna Kobalczyk 1 Mihaela van der Schaar 1
Abstract
Reward modelling from preference data is a
crucial step in aligning large language models
(LLMs) with human values, requiring robust gen-
eralisation to novel prompt-response pairs. In
this work, we propose to frame this problem in
a causal paradigm, providing the rich toolbox
of causality to identify the persistent challenges,
such as causal misidentification, preference het-
erogeneity, and confounding due to user-specific
factors. Inheriting from the literature of causal
inference, we identify key assumptions neces-
sary for reliable generalisation and contrast them
with common data collection practices. We illus-
trate failure modes of naive reward models and
demonstrate how causally-inspired approaches
can improve model robustness. Finally, we out-
line desiderata for future research and practices,
advocating targeted interventions to address inher-
ent limitations of observational data.
1. Introduction
The remarkable success of LLMs lies partially in their abil-
ity to align with human values, producing responses that
are helpful, harmless, and honest. A central method for
achieving this alignment is reinforcement learning from hu-
man feedback (RLHF), with the LLM’s behaviour shaped
by reward models derived from datasets of human prefer-
ences (Ouyang et al., 2022; Bai et al., 2022). The reward
modelling, aka preference learning stage seeks to address a
seemingly straightforward question: Given two responses
to the same prompt, which one aligns better with human
objectives? However, the challenges of this stage are of-
ten underestimated, assuming that simple regression-based
models fitted to observational datasets can generalise effec-
tively to unseen texts. Meanwhile, evidence suggests this
optimism may be misplaced (Tien et al., 2023; Skalse et al.,
1Department of Applied Mathematics and Theoretical Physics,
University of Cambridge, United Ki...
DCBM: Data-Efficient Visual Concept Bottleneck Models
ICML 2025

DCBM: Data-Efficient Visual Concept Bottleneck Models
Katharina Prasse * 1 Patrick Knab * 2 Sascha Marton 2 Christian Bartelt 2 Margret Keuper 1 3
Abstract
Concept Bottleneck Models (CBMs) enhance
the interpretability of neural networks by basing
predictions on human-understandable concepts.
However, current CBMs typically rely on con-
cept sets extracted from large language models
or extensive image corpora, limiting their effec-
tiveness in data-sparse scenarios. We propose
Data-efficient CBMs (DCBMs), which reduce the
need for large sample sizes during concept gener-
ation while preserving interpretability. DCBMs
define concepts as image regions detected by seg-
mentation or detection foundation models, allow-
ing each image to generate multiple concepts
across different granularities. Exclusively con-
taining dataset-specific concepts, DCBMs are
well suited for fine-grained classification and out-
of-distribution tasks. Attribution analysis using
Grad-CAM demonstrates that DCBMs deliver vi-
sual concepts that can be localized in test images.
By leveraging dataset-specific concepts instead
of predefined or general ones, DCBMs enhance
adaptability to new domains. The code is avail-
able at: https://github.com/KathPra/
DCBM.
1. Introduction
Deep neural networks (DNNs) achieve state-of-the-art per-
formance across various domains, yet their opaque nature
limits trust, particularly in safety-critical applications. To ad-
dress this, Explainable Artificial Intelligence (XAI) strives
to make model decisions more transparent and interpretable.
Concept Bottleneck Models (CBMs) (Koh et al., 2020) are
an inherently interpretable framework, as they base predic-
*Equal contribution
1Data and Web Science Group, Univer-
sity of Mannheim, Mannheim, Germany 2Clausthal University of
Technology, Clausthal, Germany 3Max-Planck-Institute for Infor-
matics, Saarland Informatics Campus. Correspondence to: Katha-
rina Prasse <katharina.prasse@uni-mannheim.de>, Patrick Knab
<patrick.kna...
Mastering Task Arithmetic: $\tau$Jp as a Key Indicator for Weight Disentanglement
ICLR 2025

Published as a conference paper at ICLR 2025
MASTERING TASK ARITHMETIC: τJP AS
A KEY INDICATOR FOR WEIGHT DISENTANGLEMENT
Kotaro Yoshida1,∗,†, Yuji Naraki2, Takafumi Horie3,∗, Ryosuke Yamaki 3,4,
Ryotaro Shimizu5,6, Yuki Saito5, Julian McAuley6, Hiroki Naganuma 7,8,†
1 Institute of Science Tokyo, 2 Independent Researcher, 3 Ritsumeikan University, 4 ProPlace,
5 ZOZO Research, 6 University of California San Diego, 7 Universit´e de Montr´eal, 8 Mila
ABSTRACT
Model-editing techniques using task arithmetic have rapidly gained attention.
Through task arithmetic, simply through arithmetic operations on the weights of
pre-trained and fine-tuned models create desired models, such as multi-task mod-
els, models in which specific tasks are unsolvable, or domain-transferred models.
However, task arithmetic faces challenges, such as poor reproducibility and the
high cost associated with adjusting coefficients in the arithmetic operations on
model parameters, which have limited its practical success. In this paper, we
present three key contributions in the context of task addition and task negation
within task arithmetic. First, we propose a new metric called τJp which is based
on the product of the task vector (τ) and the Jacobian of the pre-trained model with
respect to its weights. We show that τJp has a causal relationship with the inter-
ference that occurs from arithmetic operations. Second, we show that introducing
regularization to minimize τJp significantly mitigates interference between task
inference, which leads to the elimination of coefficient tuning and improved accu-
racy on each task. Third, in the context of incremental learning, we demonstrate
that our τJp regularization achieves more robust performance in environments
where access to future tasks is unavailable, thus validating the scalability of the
approach. Finally, we demonstrate that the τJp regularizer further reinforces the
performance of task arithmetic by leveraging publicly available fine-tuned mod-...
GLGENN: A Novel Parameter-Light Equivariant Neural Networks Architecture Based on Clifford Geometric Algebras
ICML 2025

GLGENN: A Novel Parameter-Light Equivariant Neural Networks
Architecture Based on Clifford Geometric Algebras
Ekaterina Filimoshina 1 2 Dmitry Shirokov 1 3
Abstract
We propose, implement, and compare with com-
petitors a new architecture of equivariant neural
networks based on geometric (Clifford) algebras:
Generalized Lipschitz Group Equivariant Neu-
ral Networks (GLGENN). These networks are
equivariant to all pseudo-orthogonal transforma-
tions, including rotations and reflections, of a vec-
tor space with any non-degenerate or degenerate
symmetric bilinear form. We propose a weight-
sharing parametrization technique that takes into
account the fundamental structures and operations
of geometric algebras. Due to this technique,
GLGENN architecture is parameter-light and has
less tendency to overfitting than baseline equiv-
ariant models. GLGENN outperforms or matches
competitors on several benchmarking equivariant
tasks, including estimation of an equivariant func-
tion and a convex hull experiment, while using
significantly fewer optimizable parameters.
1. Introduction
Equivariant neural networks are a class of neural networks
that explicitly incorporate symmetries into their architec-
ture, making them well-suited for tasks that inherently re-
quire equivariance to transformations with respect to some
group’s action (e.g., rotations, permutations, translations,
etc.). Equivariant neural networks were introduced in Cohen
& Welling, 2016 and have since been extensively developed
and applied in a wide range of tasks in computer and natural
sciences. These applications include modeling dynamical
systems (Finzi et al., 2021), particle physics (Ruhe et al.,
2023; Finzi et al., 2021), analyzing molecular properties and
1HSE University, Moscow, Russia 2 Skolkovo Institute of
Science and Technology, Moscow, Russia 3 Institute for In-
formation Transmission Problems of the Russian Academy
of Sciences, Moscow, Russia.
Correspondence to:
Ekate-
rina Filimoshina <filimoshinae...
H-Tuning: Toward Low-Cost and Efficient ECG-based Cardiovascular Disease Detection with Pre-Trained Models
ICML 2025

H-Tuning: Toward Low-Cost and Efficient ECG-based Cardiovascular Disease
Detection with Pre-Trained Models
Rushuang Zhou 1 2 Yuan-Ting Zhang 3 4 5 Yining Dong 6 2
Abstract
Fine-tuning large-scale pre-trained models pro-
vides an effective solution to alleviate the la-
bel scarcity problem in cardiovascular diseases
(CVDs) detection using electrocardiogram (ECG).
However, as the pre-trained models scale up, the
computational costs for fine-tuning and inference
become unaffordable on low-level devices de-
ployed for clinical applications.
Additionally,
maintaining the model performance under low
budgets in computational resources remains a sig-
nificant challenge. However, a comprehensive
study that can address them in a joint framework is
still lacking. Here, we propose a holistic method
(H-Tuning) for low-cost and efficient fine-tuning
of pre-trained models on downstream datasets.
Then, the inference costs of the models fine-tuned
by H-Tuning are further reduced significantly us-
ing a knowledge distillation technique. Exper-
iments on four ECG datasets demonstrate that
H-Tuning reduces the GPU memory consumption
during fine-tuning by 6.34 times while achieving
comparable CVDs detection performance to stan-
dard fine-tuning. With the knowledge distillation
technique, the model inference latency and the
memory consumption are reduced by 4.52 times
and 19.83 times. As such, the proposed joint
framework allows for the utilization of pre-trained
models with high computation efficiency and ro-
bust performance, exploring a path toward low-
cost and efficient CVDs detection. Code is avail-
able at https://github.com/KAZABANA/H-Tuning
1Department of Biomedical Engineering, City University
of Hong Kong, Hong Kong, China 2Hong Kong Center for
Cerebro-Cardiovascular Health Engineering, Hong Kong, China
3Department of Electronic Engineering, Chinese University of
Hong Kong, Hong Kong, China 4Hong Kong Institutes of Medical
Engineering, Hong Kong, China 5The AICARE Bay Lab, Gua...
Test-Time Spectrum-Aware Latent Steering for Zero-Shot Generalization in Vision-Language Models
NeurIPS 2025

Test-Time Spectrum-Aware Latent Steering for
Zero-Shot Generalization in Vision-Language Models
Konstantinos M. Dafnis∗
Department of Computer Science
Rutgers University
Dimitris N. Metaxas
Department of Computer Science
Rutgers University
Abstract
Vision–Language Models (VLMs) excel at zero-shot inference but often degrade
under test-time domain shifts. For this reason, episodic test-time adaptation strate-
gies have recently emerged as powerful techniques for adapting VLMs to a single
unlabeled image. However, existing adaptation strategies, such as test-time prompt
tuning, typically require backpropagating through large encoder weights or altering
core model components. In this work, we introduce Spectrum-Aware Test-Time
Steering (STS), a lightweight adaptation framework that extracts a spectral sub-
space from the textual embeddings to define principal semantic directions and learns
to steer latent representations in a spectrum-aware manner by adapting a small num-
ber of per-sample shift parameters to minimize entropy across augmented views.
STS operates entirely at inference in the latent space, without backpropagation
through or modification of the frozen encoders. Building on standard evaluation
protocols, our comprehensive experiments demonstrate that STS largely surpasses
or compares favorably against state-of-the-art test-time adaptation methods, while
introducing only a handful of additional parameters and achieving inference speeds
up to 8× faster with a 12× smaller memory footprint than conventional test-time
prompt tuning. The code is available at https://github.com/kdafnis/STS.
1
Introduction
Vision-Language Models (VLMs), such as CLIP [30], have marked a paradigm shift in artificial
intelligence, demonstrating remarkable zero-shot generalization capabilities across a multitude of
downstream visual tasks. By learning rich joint representations from vast quantities of image-text
data, these models can often perform tasks like image classification with...
Benign Overfitting in Token Selection of Attention Mechanism
ICML 2025

Benign Overfitting in Token Selection of Attention Mechanism
Keitaro Sakamoto 1 Issei Sato 1
Abstract
Attention mechanism is a fundamental component
of the transformer model and plays a significant
role in its success. However, the theoretical under-
standing of how attention learns to select tokens
is still an emerging area of research. In this work,
we study the training dynamics and generalization
ability of the attention mechanism under classifi-
cation problems with label noise. We show that,
with the characterization of signal-to-noise ratio
(SNR), the token selection of attention mechanism
achieves “benign overfitting”, i.e., maintaining
high generalization performance despite fitting
label noise. Our work also demonstrates an inter-
esting delayed acquisition of generalization after
an initial phase of overfitting. Finally, we provide
experiments to support our theoretical analysis
using both synthetic and real-world datasets.
1. Introduction
The transformer models (Vaswani et al., 2017) have greatly
succeeded across a wide range of fields, including natural
language (Devlin et al., 2018; Brown et al., 2020), vision
(Dosovitskiy et al., 2021; Touvron et al., 2021), and have be-
come a foundational architecture in modern machine learn-
ing. The key component that characterizes the transformer
model is the attention mechanism, which was originally
introduced in recurrent neural network (RNN) and long
short-term memory (LSTM) to capture the long-range struc-
ture of sequence (Bahdanau, 2014; Xu, 2015). The attention
architecture can process variable-length input sequences
and flexibly select important tokens based on the input. The
seminal work on the training dynamics of this token selec-
tion mechanism (Tarzanagh et al., 2023a;b; Vasudeva et al.,
2024; Sheen et al., 2024) studied the implicit bias of gradi-
ent descent to the max-margin token separator. However,
1Department of Computer Science, The University of
Tokyo, Tokyo, Japan. Correspondence to: Keitaro...
Joint Relational Database Generation via Graph-Conditional Diffusion Models
NeurIPS 2025

Joint Relational Database Generation via
Graph-Conditional Diffusion Models
Mohamed Amine Ketata, David Lüdke, Leo Schwinn, Stephan Günnemann
School of Computation, Information and Technology & Munich Data Science Institute
Technical University of Munich, Germany
Correspondence to: a.ketata@tum.de
Abstract
Building generative models for relational databases (RDBs) is important for many
applications, such as privacy-preserving data release and augmenting real datasets.
However, most prior works either focus on single-table generation or adapt single-
table models to the multi-table setting by relying on autoregressive factorizations
and sequential generation. These approaches limit parallelism, restrict flexibility in
downstream applications, and compound errors due to commonly made conditional
independence assumptions. In this paper, we propose a fundamentally different ap-
proach: jointly modeling all tables in an RDB without imposing any table order. By
using a natural graph representation of RDBs, we propose the Graph-Conditional
Relational Diffusion Model (GRDM), which leverages a graph neural network
to jointly denoise row attributes and capture complex inter-table dependencies.
Extensive experiments on six real-world RDBs demonstrate that our approach sub-
stantially outperforms autoregressive baselines in modeling multi-hop inter-table
correlations and achieves state-of-the-art performance on single-table fidelity met-
rics. Our code is available at https://github.com/ketatam/rdb-diffusion.
1
Introduction
Table A
Table B
Table C
Step 1
Autoregressive
Joint (Ours)
Limiting table order
   
Sequential generation
   
Simplifying assumptions
No table order
     Parallel generation
     Joint modeling of all rows
Step 2
Step 3
Table A
Table B
Table C
Figure 1: Comparison of autoregressive
and joint relational database generation.
Relational databases (RDBs), which organize data into
multiple interlinked tables, are the most widely used
data management system, estim...
The Belief State Transformer
ICLR 2025

Published as a conference paper at ICLR 2025
THE BELIEF STATE TRANSFORMER
Edward S. Hu1,2
Kwangjun Ahn1
Qinghua Liu1
Haoran Xu1,3
Manan Tomar1,4
Ada Langford1
Dinesh Jayaraman2
Alex Lamb1
John Langford1
1Microsoft Research
2University of Pennsylvania
3UT Austin
4 University of Alberta
ABSTRACT
We introduce the “Belief State Transformer”, a next-token predictor that takes both
a prefix and suffix as inputs, with a novel objective of predicting both the next token
for the prefix and the previous token for the suffix. The Belief State Transformer
effectively learns to solve challenging problems that conventional forward-only
transformers struggle with, in a domain-independent fashion. Key to this success is
learning a compact belief state that captures all relevant information necessary for
accurate predictions. Empirical ablations show that each component of the model is
essential in difficult scenarios where standard Transformers fall short. For the task
of story writing with known prefixes and suffixes, our approach outperforms the
Fill-in-the-Middle method for reaching known goals and demonstrates improved
performance even when the goals are unknown. Altogether, the Belief State
Transformer enables more efficient goal-conditioned decoding, better test-time
inference, and high-quality text representations on small scale problems. Website:
https://edwhu.github.io/bst-website
1
INTRODUCTION
Transformer models (Vaswani et al., 2017) have created a revolution in language modeling (Achiam
et al., 2023) with the capability to generate language with many emergent properties at large scale.
Examining these models for flaws in the pursuit of further progress, it’s notable that they struggle
with planning-heavy problems (Bubeck et al., 2023; Momennejad et al., 2024). How can we modify
the architecture, objectives, and algorithms to create a model more capable of reaching goals?
To make progress, we propose the new Belief State Transformer (BST) architecture and objective
in S...
Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces
ICML 2025

Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces
Kevin Rojas * 1 2 Yuchen Zhu * 1 2 Sichen Zhu 2 Felix X.-F. Ye 3 Molei Tao 1 2
Abstract
Diffusion models have demonstrated remarkable
performance in generating unimodal data across
various tasks, including image, video, and text
generation. On the contrary, the joint generation
of multimodal data through diffusion models is
still in the early stages of exploration. Existing
approaches heavily rely on external preprocess-
ing protocols, such as tokenizers and variational
autoencoders, to harmonize varied data represen-
tations into a unified, unimodal format.
This
process heavily demands the high accuracy of
encoders and decoders, which can be problem-
atic for applications with limited data. To lift
this restriction, we propose a novel framework
for building multimodal diffusion models on arbi-
trary state spaces, enabling native generation of
coupled data across different modalities. By in-
troducing an innovative decoupled noise schedule
for each modality, we enable both unconditional
and modality-conditioned generation within a sin-
gle model simultaneously. We empirically vali-
date our approach for text-image generation and
mixed-type tabular data synthesis, demonstrating
that it achieves competitive performance. Code is
available at Diffuse-Everything.
1. Introduction
Recent years have witnessed the tremendous success of
diffusion generative models in various applications. The
seminal works of continuous diffusion models on Euclidean
spaces (Sohl-Dickstein et al., 2015; Song et al., 2020; Ho
et al., 2020) have led to state-of-the-art methods for tasks
such as image generation (Dhariwal & Nichol, 2021; Bao
et al., 2023a; Karras et al., 2022; 2024b), video generation
*Equal contribution, random order by coin flip 1School of Math-
ematics, Georgia Institute of Technology, Atlanta, GA 2Machine
Learning Center, Georgia Institute of Technology, Atlanta, GA
3Department of Mathematics & Statisti...