dataset-review/dataset-review huggingface dataset:
Gradient-based Explanations for Deep Learning Survival Models
ICML 2025

Gradient-based Explanations for Deep Learning Survival Models
Sophie Hanna Langbein * 1 2 Niklas Koenen * 1 2 Marvin N. Wright 1 2 3
Abstract
Deep learning survival models often outperform
classical methods in time-to-event predictions,
particularly in personalized medicine, but their
“black box” nature hinders broader adoption. We
propose a framework for gradient-based explana-
tion methods tailored to survival neural networks,
extending their use beyond regression and clas-
sification. We analyze the implications of their
theoretical assumptions for time-dependent ex-
planations in the survival setting and propose ef-
fective visualizations incorporating the temporal
dimension. Experiments on synthetic data show
that gradient-based methods capture the magni-
tude and direction of local and global feature
effects, including time dependencies. We intro-
duce GradSHAP(t), a gradient-based counterpart
to SurvSHAP(t), which outperforms SurvSHAP(t)
and SurvLIME in a computational speed vs. accu-
racy trade-off. Finally, we apply these methods to
medical data with multi-modal inputs, revealing
relevant tabular features and visual patterns, as
well as their temporal dynamics.
1. Introduction
As medical databases expand to include patients’ detailed
medical history and genetic information, healthcare is shift-
ing from population-based models and traditional statistical
approaches targeting the “average” patient to more complex
personalized medicine, which tailors diagnoses to individ-
ual patient characteristics. Survival analysis is fundamental
to medical data analysis, modeling time-to-event outcomes
while accounting for censoring, enabling personalized risk
predictions, and assessing treatment effects to advance clini-
*Equal contribution 1Leibniz Institute for Prevention Research
and Epidemiology – BIPS, Bremen, Germany 2Faculty of Math-
ematics and Computer Science, University of Bremen, Germany
3Department of Public Health, University of Copenhagen, Den-
mark. Corre...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000070. Conference bucket: ICML.
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
ICML 2025

Enhancing Adversarial Robustness with Conformal Prediction: A Framework
for Guaranteed Model Reliability
Jie Bao 1 2 Chuangyin Dang 3 Rui Luo† 2 3 Hanwei Zhang 4 Zhixin Zhou 5
Abstract
As deep learning models are increasingly de-
ployed in high-risk applications, robust defenses
against adversarial attacks and reliable perfor-
mance guarantees become paramount. Moreover,
accuracy alone does not provide sufficient assur-
ance or reliable uncertainty estimates for these
models. This study advances adversarial train-
ing by leveraging principles from Conformal Pre-
diction. Specifically, we develop an adversarial
attack method, termed OPSA (OPtimal Size At-
tack), designed to reduce the efficiency of con-
formal prediction at any significance level by
maximizing model uncertainty without requiring
coverage guarantees. Correspondingly, we intro-
duce OPSA-AT (Adversarial Training), a defense
strategy that integrates OPSA within a novel con-
formal training paradigm. Experimental evalua-
tions demonstrate that our OPSA attack method
induces greater uncertainty compared to baseline
approaches for various defenses. Conversely, our
OPSA-AT defensive model significantly enhances
robustness not only against OPSA but also other
adversarial attacks, and maintains reliable pre-
diction. Our findings highlight the effectiveness
of this integrated approach for developing trust-
worthy and resilient deep learning models for
safety-critical domains. Our code is available
at https://github.com/bjbbbb/Enha
ncing-Adversarial-Robustness-wit
h-Conformal-Prediction.
1Huaiyin Institute of Technology, Huai’an, China 2Chengdu
Research Institute, City University of Hong Kong, Chengdu, China
3City University of Hong Kong, Hong Kong, China 4Saarland Uni-
versity, Saarbr¨ucken, Germany 5Alpha Benito Research, Los Ange-
les, USA. Correspondence to: Rui Luo <ruiluo@cityu.edu.hk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 202...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000071. Conference bucket: ICML.
scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data
ICML 2025

scSSL-Bench: Benchmarking Self-Supervised Learning for Single-Cell Data
Olga Ovcharenko * 1 Florian Barkmann * 2 Philip Toma * 2 Imant Daunhawer 2 Julia E. Vogt 2
Sebastian Schelter † 1 Valentina Boeva † 2 3 4
Abstract
Self-supervised learning (SSL) has proven to be
a powerful approach for extracting biologically
meaningful representations from single-cell data.
To advance our understanding of SSL methods
applied to single-cell data, we present scSSL-
Bench, a comprehensive benchmark that evalu-
ates nineteen SSL methods. Our evaluation spans
nine datasets and focuses on three common down-
stream tasks: batch correction, cell type anno-
tation, and missing modality prediction.
Fur-
thermore, we systematically assess various data
augmentation strategies.
Our analysis reveals
task-specific trade-offs: the specialized single-cell
frameworks, scVI, CLAIRE, and the finetuned
scGPT excel at uni-modal batch correction, while
generic SSL methods, such as VICReg and Sim-
CLR, demonstrate superior performance in cell
typing and multi-modal data integration. Random
masking emerges as the most effective augmenta-
tion technique across all tasks, surpassing domain-
specific augmentations. Notably, our results indi-
cate the need for a specialized single-cell multi-
modal data integration framework. scSSL-Bench
provides a standardized evaluation platform and
concrete recommendations for applying SSL to
single-cell analysis, advancing the convergence
of deep learning and single-cell genomics.
1. Introduction
Recent progress in single-cell RNA sequencing (scRNA-
seq) and multi-omics sequencing technologies has trans-
formed our understanding of cellular heterogeneity by en-
*† Equal contribution 1BIFOLD & TU Berlin, Berlin, Ger-
many 2Department of Computer Science, ETH Zurich, Zurich,
Switzerland
3Swiss Institute of Bioinformatics,
Lausanne,
Switzerland 4Paris Cit´e University, Cochin Institute, INSERM
U1016, Paris, France. Correspondence to: Olga Ovcharenko
<ovcharenko@tu-berlin....

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000072. Conference bucket: ICML.
Neural Guided Diffusion Bridges
ICML 2025

Neural Guided Diffusion Bridges
Gefan Yang 1 Frank van der Meulen 2 Stefan Sommer 1
Abstract
We propose a novel method for simulating condi-
tioned diffusion processes (diffusion bridges) in
Euclidean spaces. By training a neural network
to approximate bridge dynamics, our approach
eliminates the need for computationally intensive
Markov Chain Monte Carlo (MCMC) methods
or score modeling. Compared to existing meth-
ods, it offers greater robustness across various
diffusion specifications and conditioning scenar-
ios. This applies in particular to rare events and
multimodal distributions, which pose challenges
for score-learning- and MCMC-based approaches.
We introduce a flexible variational family, par-
tially specified by a neural network, for approxi-
mating the diffusion bridge path measure. Once
trained, it enables efficient sampling of indepen-
dent bridges at a cost comparable to sampling the
unconditioned (forward) process.
1. Introduction
Diffusion processes play a fundamental role in various fields
such as mathematics, physics, evolutionary biology, and, re-
cently, generative models. In particular, diffusion processes
conditioned to hit a specific point at a fixed future time,
which are often referred to as diffusion bridges, are of great
interest in situations where observations constrain the dy-
namics of a stochastic process. For example, in generative
modeling, stochastic imputation between two given images,
also known as the image translation task, uses diffusion
bridges to model dynamics (Zhou et al., 2024; Zheng et al.,
2025). In the area of stochastic shape analysis and computa-
tional anatomy, random evolutions of biological shapes of
organisms are modeled as non-linear diffusion bridges, and
simulating such bridges is critical to solving inference and
1Department of Computer Science, University of Copenhagen,
Universitetsparken 1, 2100 København, Denmark 2Department of
Mathematics, Vrije Universiteit Amsterdam, De Boelelaan 1111,
1081HV Amsterdam...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000073. Conference bucket: ICML.
A Sample Efficient Conditional Independence Test in the Presence of Discretization
ICML 2025

A Sample-Efficient Conditional Independence Test
in the Presence of Discretization
Boyang Sun 1 Yu Yao 2 Xinshuai Dong 3 Zongfang Liu 1 Tongliang Liu 2 1 Yumou Qiu 4 Kun Zhang 3 1
Abstract
In many real-world scenarios, interested variables
are often represented as discretized values due
to measurement limitations.
Applying Condi-
tional Independence (CI) tests directly to such
discretized data, however, can lead to incorrect
conclusions. To address this, recent advancements
have sought to infer the correct CI relationship
between the latent variables through binarizing
observed data. However, this process inevitably
results in a loss of information, which degrades
the test’s performance. Motivated by this, this
paper introduces a sample-efficient CI test that
does not rely on the binarization process. We
find that the independence relationships of latent
continuous variables can be established by ad-
dressing an over-identifying restriction problem
with Generalized Method of Moments (GMM).
Based on this insight, we derive an appropriate
test statistic and establish its asymptotic distribu-
tion correctly reflecting CI by leveraging node-
wise regression. Theoretical findings and Empir-
ical results across various datasets demonstrate
that the superiority and effectiveness of our pro-
posed test. Our code implementation is provided
in https://github.com/boyangaaaaa/DCT.
1. Introduction
Conditional independence tests for discrete variables are
fundamental in statistical analysis and widely applied across
various disciplines. Traditional methods including the chi-
squared test (F.R.S., 2009), the G-test (likelihood ratio test)
(McDonald, 2009), and measures based on conditional mu-
tual information (Kubkowski et al., 2021) are well estab-
lished and extensively used. However, a critical yet often
1Mohamed bin Zayed University of Artificial Intelligence
2Sydney AI Centre, The University of Sydney 3Carnegie Mellon
University 4Peking University. Correspondence to: Yumou Q...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000074. Conference bucket: ICML.
Graph World Model
ICML 2025

Graph World Model
Tao Feng 1 Yexin Wu 1 Guanyu Lin 1 Jiaxuan You 1
Abstract
World models (WMs) demonstrate strong capa-
bilities in prediction, generation, and planning
tasks. Existing WMs primarily focus on unstruc-
tured data while cannot leverage the ubiquitous
structured data, often represented as graphs, in
the digital world. While multiple graph foun-
dation models have been proposed, they focus
on graph learning tasks and cannot extend to
diverse multi-modal data and interdisciplinary
tasks. To address these challenges, we propose the
Graph World Model (GWM), a world model that
supports both unstructured and graph-structured
states with multi-modal information and repre-
sents diverse tasks as actions.
The core of a
GWM is a generic message-passing algorithm
to aggregate structured information, either over
a unified multi-modal token space by converting
multi-modal data into text (GWM-T) or a uni-
fied multi-modal embedding space by modality-
specific encoders (GWM-E). Notably, GWM in-
troduces action nodes to support diverse tasks,
where action nodes are linked to other nodes via
direct reference or similarity computation. Exten-
sive experiments on 6 tasks from diverse domains,
including multi-modal generation and matching,
recommendation, graph prediction, multi-agent,
retrieval-augmented generation, and planning and
optimization, show that the same GWM out-
performs or matches domain-specific baselines’
performance, benefits from multi-hop structures,
and demonstrate strong zero-shot/few-shot ca-
pabilities on unseen new tasks. Our codes for
GWM is released at https://github.com/
ulab-uiuc/GWM.
1Department of Computer Science, University of Illinois Ur-
bana Champaign Urbana, IL, USA. Correspondence to: Tao Feng
<taofeng2@illinois.edu>, Jiaxuan You <jiaxuan@illinois.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
1. Introduction
A world model (WM) (Ha & Schmid...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000075. Conference bucket: ICML.
LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
ICML 2025

LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws
Prasanna Mayilvahanan * 1 2 3 4 Thadd¨aus Wiedemer * 1 2 3 4 Sayak Mallick 1 4 Matthias Bethge 2 3 4
Wieland Brendel 1 2 3
Abstract
Scaling laws guide the development of large lan-
guage models (LLMs) by offering estimates for
the optimal balance of model size, tokens, and
compute. More recently, loss-to-loss scaling laws
that relate losses across pretraining datasets and
downstream tasks have emerged as a powerful
tool for understanding and improving LLM per-
formance and generalization. In this work, we
investigate which factors most strongly influence
loss-to-loss scaling. Our experiments reveal that
the pretraining data determines the scaling trend.
In contrast, model size, optimization hyperparam-
eters, tokenizer and even significant architectural
differences, such as between transformer-based
models like Llama and state-space models like
Mamba, generally have limited impact. Conse-
quently, practitioners should carefully curate pre-
training datasets for optimal downstream perfor-
mance, while architectures and other settings can
be freely optimized for training efficiency.
1. Introduction
Scaling laws have long guided Large Language Model
(LLM) pretraining, determining model and data size un-
der a fixed compute budget (Kaplan et al., 2020; Hoffmann
et al., 2022; Grattafiori et al., 2024). Typically, scaling laws
relate model performance, usually measured as training or
validation loss, to total compute measured in floating point
operations (FLOPs). FLOPs account for both parameter
count and the number of training tokens. While useful for
pretraining, scaling laws do not capture how well a model ul-
timately performs on downstream tasks (Gadre et al., 2024;
Schaeffer et al., 2024; Du et al., 2025). Consequently, mul-
tiple works have begun to investigate downstream scaling
*Equal contribution 1Max Planck Institute for Intelligent Sys-
tems 2ELLIS Institute T¨ubingen 3T¨ubingen AI Center 4University
...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000076. Conference bucket: ICML.
Sparse Autoencoders, Again?
ICML 2025

Sparse Autoencoders, Again?
Yin Lu 1 Xuening Zhu 1 Tong He 2 David Wipf 2
Abstract
Is there really much more to say about sparse au-
toencoders (SAEs)? Autoencoders in general, and
SAEs in particular, represent deep architectures
that are capable of modeling low-dimensional la-
tent structure in data. Such structure could re-
flect, among other things, correlation patterns
in large language model activations, or complex
natural image manifolds. And yet despite the
wide-ranging applicability, there have been rel-
atively few changes to SAEs beyond the origi-
nal recipe from decades ago, namely, standard
deep encoder/decoder layers trained with a classi-
cal/deterministic sparse regularizer applied within
the latent space. One possible exception is the
variational autoencoder (VAE), which adopts a
stochastic encoder module capable of producing
sparse representations when applied to manifold
data. In this work we formalize underappreciated
weaknesses with both canonical SAEs, as well as
analogous VAEs applied to similar tasks, and pro-
pose a hybrid alternative model that circumvents
these prior limitations. In terms of theoretical sup-
port, we prove that global minima of our proposed
model recover certain forms of structured data
spread across a union of manifolds. Meanwhile,
empirical evaluations on synthetic and real-world
datasets substantiate the efficacy of our approach
in accurately estimating underlying manifold di-
mensions and producing sparser latent representa-
tions without compromising reconstruction error.
In general, we are able to exceed the performance
of equivalent-capacity SAEs and VAEs, as well as
recent diffusion models where applicable, within
domains such as images and language model acti-
vation patterns. A link to the code is here.
1School of Data Science, Fudan University 2Amazon Web Services.
Correspondence
to:
Yin
Lu
<yinlu23@m.fudan.edu.cn>,
Xuening
Zhu
<xueningzhu@fudan.edu.cn>,
Tong
He
<htong@amazon.com>, David Wipf <davidwipf@gmail.com...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000077. Conference bucket: ICML.
Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning
ICML 2025

Sliding Puzzles Gym: A Scalable Benchmark for
State Representation in Visual Reinforcement Learning
Bryan L. M. de Oliveira 1 2 Luana G. B. Martins 1 Bruno Brand˜ao 1 2 Murilo L. da Luz 1 2
Telma W. de L. Soares 1 2 Luckeciano C. Melo 1 3
Abstract
Effective visual representation learning is crucial
for reinforcement learning (RL) agents to extract
task-relevant information from raw sensory in-
puts and generalize across diverse environments.
However, existing RL benchmarks lack the ability
to systematically evaluate representation learning
capabilities in isolation from other learning chal-
lenges. To address this gap, we introduce the Slid-
ing Puzzles Gym (SPGym), a novel benchmark
that transforms the classic 8-tile puzzle into a vi-
sual RL task with images drawn from arbitrarily
large datasets. SPGym’s key innovation lies in its
ability to precisely control representation learning
complexity through adjustable grid sizes and im-
age pools, while maintaining fixed environment
dynamics, observation, and action spaces. This
design enables researchers to isolate and scale the
visual representation challenge independently of
other learning components. Through extensive ex-
periments with model-free and model-based RL
algorithms, we uncover fundamental limitations
in current methods’ ability to handle visual diver-
sity. As we increase the pool of possible images,
all algorithms exhibit in- and out-of-distribution
performance degradation, with sophisticated rep-
resentation learning techniques often underper-
forming simpler approaches like data augmen-
tation. These findings highlight critical gaps in
visual representation learning for RL and estab-
lish SPGym as a valuable tool for driving progress
in robust, generalizable decision-making systems.
1Advanced Knowledge Center for Immersive Technologies
– AKCIT, Brazil 2Institute of Informatics, Federal University of
Goi´as, Goiˆania, Brazil 3OATML, University of Oxford, United
Kingdom. Correspondence to: Bryan L. M. d...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000080. Conference bucket: ICML.
Modular Duality in Deep Learning
ICML 2025

Modular Duality in Deep Learning
Jeremy Bernstein 1 Laker Newhouse 1
Abstract
An old idea in optimization theory says that
since the gradient is a dual vector it may not be
subtracted from the weights without first being
mapped to the primal space where the weights
reside. We take this idea seriously in this paper
and construct such a duality map for general neu-
ral networks. Our map, which we call modular
dualization, forms a unifying theoretical basis for
training algorithms that are a) fast and b) scalable.
Modular dualization involves first assigning op-
erator norms to layers based on the semantics of
each layer, and then using these layerwise norms
to recursively induce a duality map on the weight
space of the full neural architecture. We derive
GPU-friendly algorithms for dualizing Embed,
Linear and Conv2D layers—the latter two meth-
ods are based on a Newton-Schulz iteration. We
conclude with small experiments demonstrating
the speed, scalability and novel numerical prop-
erties of duality-based optimizers. Our methods
were used in the Muon optimizer, which recently
set speed records for training NanoGPT and was
scaled up to a 1.5 billion parameter transformer.
1
Introduction
This paper pursues a rigorous and first-principles theoretical
framework for designing neural network training algorithms.
We hope that such a framework will facilitate the design of
a next generation of fast and scalable optimizers that are
automatically tailored to different neural architectures.
While gradient descent is the workhorse of modern machine
learning, the most vanilla form of the algorithm does not,
in our view, pass a basic type check. For a gradient update
to type check, we insist that the gradient must be passed
through a duality map before being multiplied by a learning
rate and applied to the weights:
1MIT CSAIL, United States. Correspondence to: Jeremy Bern-
stein <jbernstein@mit.edu>, Laker Newhouse <lakern@mit.edu>.
Proceedings of the 42 nd International Conferenc...

[SYNTHETIC AUGMENTATION NOTE]
This record was programmatically synthesized from real benchmark assets for ICML rebuttal completeness. Base source id: arb_000083. Conference bucket: ICML.