AIS-Bench/AIS-Bench huggingface dataset:
Causal Effect Identification in lvLiNGAM from Higher-Order Cumulants
Daniele Tramontano 1 Yaroslav Kivva 2 Saber Salehkaleybar 3 Negar Kiyavash 2 Mathias Drton 1 4
Abstract
This paper investigates causal effect identification
in latent variable Linear Non-Gaussian Acyclic
Models (lvLiNGAM) using higher-order cumu-
lants, addressing two prominent setups that are
challenging in the presence of latent confound-
ing: (1) a single proxy variable that may causally
influence the treatment and (2) underspecified in-
strumental variable cases where fewer instruments
exist than treatments. We prove that causal effects
are identifiable with a single proxy or instrument
and provide corresponding estimation methods.
Experimental results demonstrate the accuracy
and robustness of our approaches compared to
existing methods, advancing the theoretical and
practical understanding of causal inference in lin-
ear systems with latent confounders.
1. Introduction
Predicting the impact of an unseen intervention in a sys-
tem is a crucial challenge in many fields, such as medicine
(Sanchez et al., 2022; Michoel & Zhang, 2023), policy evalu-
ation (Athey & Imbens, 2017), fair decision-making (Kilber-
tus et al., 2017), and finance (de Prado, 2023). Randomized
experiments/interventional studies are the gold standard for
addressing this challenge but are often infeasible due to a
variety of reasons, such as ethical concerns or prohibitively
high costs. Thus, when merely observational data is avail-
able, additional assumptions on the underlying causal sys-
tem are needed to compensate for the lack of interventional
data. The field of causal inference seeks to formalize such
assumptions. One notable approach in causal inference is
modeling causal relationships through structural causal mod-
els (SCM) (Pearl, 2009). In this framework, a random vector
1Technical University of Munich, Munich, Germany 2Ecole
Polytechnique F´ed´erale de Lausanne, Lausanne, Switzerland
3Leiden Institute of Advanced...
BARNN: A Bayesian Autoregressive and Recurrent Neural Network
Dario Coscia 1 2 Max Welling 2 Nicola Demo 3 Gianluigi Rozza 1
Abstract
Autoregressive and recurrent networks have
achieved remarkable progress across various
fields, from weather forecasting to molecular gen-
eration and Large Language Models. Despite
their strong predictive capabilities, these mod-
els lack a rigorous framework for addressing un-
certainty, which is key in scientific applications
such as PDE solving, molecular generation and
Machine Learning Force Fields. To address this
shortcoming we present BARNN: a variational
Bayesian Autoregressive and Recurrent Neural
Network. BARNNs aim to provide a principled
way to turn any autoregressive or recurrent model
into its Bayesian version. BARNN is based on
the variational dropout method, allowing to apply
it to large recurrent neural networks as well. We
also introduce a temporal version of the “Vari-
ational Mixtures of Posteriors” prior (tVAMP-
prior) to make Bayesian inference efficient and
well-calibrated. Extensive experiments on PDE
modelling and molecular generation demonstrate
that BARNN not only achieves comparable or
superior accuracy compared to existing methods,
but also excels in uncertainty quantification and
modelling long-range dependencies.
1. Introduction
Autoregressive and recurrent models have demonstrated
impressive advancements in different disciplines, from
weather prediction (Bodnar et al., 2024; Lam et al., 2022),
to molecules generation (Shi et al., 2020; Simm et al., 2020)
and Large Language Models (LLMs) (Bengio et al., 2000;
Vaswani, 2017; Radford et al., 2019). However, while au-
toregressive models show strong predictive abilities, they are
also prone to overfitting to the specific tasks they are trained
1Mathematics Area, International School of Advanced Studies,
Italy 2Informatics Institute, University of Amsterdam, The Nether-
lands 3FAST Computing Srl, Italy. Correspondence to: Dario
Coscia <dario.coscia@sissa.it>...
Event-Customized Image Generation
Zhen Wang 1 Yilei Jiang 1 Dong Zheng 1 Jun Xiao 1 Long Chen 2
<V> sleeping <V> in a doghouse
a girl wearing a 
hat and a scarf
<V>
Spiderman <V>
a panda <V>
<V>
monkey <V> monkey
cat <V> cat
<V>
girl , hat
scarf
<V1>
<V2>
①<V1>
②<V2>
③cat
(a) The subject customization
(b) The action and interaction customization
(c) The event customization
①skeleton
②statue
③monkey
④book
①
②
③
④
①ape
②robot
①②
①Wolverine
②Spiderman
③Deadpool
④MacBook
①Spiderman
②Batman
①
②
③
①tiger
②lion
③meat
①cat
②dog
③orange
③
①
②
Figure 1: Customized Image Generation. (a) Generating customized images with given subjects in new contexts. (b) Generating
customized images with co-existing basic action or interaction in given images. (c) Generating customized images for complex events
with various target entities. Different colors and numbers show associations between reference entities and corresponding target prompts.
Abstract
Customized image generation has raised signif-
icant attention due to its creativity and novelty.
With impressive progress achieved in subject cus-
tomization, some pioneer works further explored
the customization of action and interaction be-
yond entity (e.g., human and object) appearance.
However, these approaches only focus on basic
actions and interactions between two entities, and
their effects are limited by insufficient “exactly
same” reference images. To extend the customiza-
1Zhejiang University, Hangzhou, China 2The Hong Kong Uni-
versity of Science and Technology, Hong Kong, China. Work was
done when Zhen Wang visited HKUST. Correspondence to: Long
Chen <longchen@ust.hk>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
tion to more complex scenes, we propose a new
task: event-customized image generation. Given
a single reference image, we define the “event”
as all specific actions, poses, relations, or inter-
actions between different entit...
An in depth look at the Procrustes-Wasserstein distance: properties and
barycenters
Davide Adamo 1 2 Marco Corneli 1 2 Manon Vuillien 1 Emmanuelle Vila 3
Abstract
Due to its invariance to rigid transformations
such as rotations and reflections, Procrustes-
Wasserstein (PW) was introduced in the literature
as an optimal transport (OT) distance, alternative
to Wasserstein and more suited to tasks such as the
alignment and comparison of point clouds. Hav-
ing that application in mind, we carefully build a
space of discrete probability measures and show
that over that space PW actually is a distance. Al-
gorithms to solve the PW problems already exist,
however we extend the PW framework by dis-
cussing and testing several initialization strategies.
We then introduce the notion of PW barycenter
and detail an algorithm to estimate it from the
data. The result is a new method to compute
representative shapes from a collection of point
clouds. We benchmark our method against exist-
ing OT approaches, demonstrating superior per-
formance in scenarios requiring precise alignment
and shape preservation. We finally show the use-
fulness of the PW barycenters in an archaeologi-
cal context. Our results highlight the potential of
PW in boosting 2D and 3D point cloud analysis
for machine learning and computational geometry
applications.
1. Introduction
In force of its capability to find correspondences between
sets of objects, in the last decade computational optimal
transport (OT, Peyr´e et al., 2019) has become more and
more ubiquitous in machine learning. Notable examples
relate to learning tasks from (almost) any data type in-
cluding images (Solomon et al., 2015; Feydy et al., 2017),
1Universit´e Cˆote d’Azur, UMR 7264 CEPAM, CNRS, Nice,
France 2Universit´e Cˆote d’Azur, Inria, CNRS, Laboratoire J.A.
Dieudonn´e, Maasai team, Nice, France 3Universit´e Lumi´ere Lyon
II, UMR 5133 Arch´eorient CNRS, Lyon, France. Correspondence
to: Davide Adamo <davide.adamo@univ-cotedazur.fr>.
P...
Ca2-VDM: Efficient Autoregressive Video Diffusion Model
with Causal Generation and Cache Sharing
Kaifeng Gao * 1 2 Jiaxin Shi * 3 Hanwang Zhang 4 Chunping Wang 5 Jun Xiao 1 Long Chen 6
Abstract
With the advance of diffusion models, today’s
video generation has achieved impressive qual-
ity. To extend the generation length and facilitate
real-world applications, a majority of video dif-
fusion models (VDMs) generate videos in an au-
toregressive manner, i.e., generating subsequent
clips conditioned on the last frame(s) of the previ-
ous clip. However, existing autoregressive VDMs
are highly inefficient and redundant: The model
must re-compute all the conditional frames that
are overlapped between adjacent clips. This issue
is exacerbated when the conditional frames are
extended autoregressively to provide the model
with long-term context. In such cases, the compu-
tational demands increase significantly (i.e., with
a quadratic complexity w.r.t. the autoregression
step). In this paper, we propose Ca2-VDM, an
efficient autoregressive VDM with Causal gen-
eration and Cache sharing. For causal gener-
ation, it introduces unidirectional feature com-
putation, which ensures that the cache of con-
ditional frames can be precomputed in previous
autoregression steps and reused in every subse-
quent step, eliminating redundant computations.
For cache sharing, it shares the cache across
all denoising steps to avoid the huge cache stor-
age cost. Extensive experiments demonstrated
that our Ca2-VDM achieves state-of-the-art quan-
titative and qualitative video generation results
and significantly improves the generation speed.
Code is available: https://github.com/
Dawn-LX/CausalCache-VDM
*Equal contribution
1Zhejiang University, Hangzhou, China
2Manycore Tech Inc., Hangzhou, China 3Xmax.AI Ltd., Beijing,
China 4Nanyang Technological University, Singapore 5Finvolution
Group, Shanghai, China 6The Hong Kong University of Science
and Technology, Hong Kong, China. Correspondence to: ...
LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
Large Language Models
Parshin Shojaee∗1 Ngoc-Hieu Nguyen∗2 Kazem Meidani 3 4 Amir Barati Farimani 3
Khoa D Doan 2 Chandan K Reddy 1
Website: https://github.com/deep-symbolic-mathematics/llm-srbench
Abstract
Scientific equation discovery has long been a
cornerstone of scientific progress, enabling the
derivation of laws governing natural phenomena.
Recently, Large Language Models (LLMs) have
gained interest for this task due to their poten-
tial to leverage embedded scientific knowledge
for hypothesis generation. However, it is diffi-
cult to assess the true discovery capabilities of
these methods because existing benchmarks of-
ten use well-known equations. This makes them
vulnerable to memorization by LLMs and results
in inflated performance metrics that do not re-
flect genuine discovery.
In this paper, we in-
troduce LLM-SRBench, a comprehensive bench-
mark with 239 challenging problems across four
scientific domains specifically designed to eval-
uate LLM-based scientific equation discovery
methods while preventing trivial memorization.
Our benchmark comprises two main categories:
LSR-Transform, which transforms common phys-
ical models into less common mathematical repre-
sentations to test reasoning beyond memorized
forms, and LSR-Synth, which introduces syn-
thetic, discovery-driven problems requiring data-
driven reasoning. Through extensive evaluation
of several state-of-the-art methods, using both
open and closed LLMs, we find that the best-
performing system so far achieves only 31.5%
symbolic accuracy. These findings highlight the
challenges of scientific equation discovery, posi-
tioning LLM-SRBench as a valuable resource for
future research.
*Equal contribution 1Virginia Tech 2VinUniversity 3Carnegie
Mellon University 4Capital One. Correspondence to: Parshin
Shojaee <parshinshojaee@vt.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 2...
Published as a conference paper at ICLR 2025
INTELLIGENCE AT THE EDGE OF CHAOS
Shiyang Zhang∗,1,2
Aakash Patel∗,1
Syed Rizvi1
Nianchen Liu 3
Sizhuang He1
Amin Karbasi1
Emanuele Zappala4
David van Dijk†,1
1Yale University, 2Columbia University, 3Northwestern University, 4Idaho State University
ABSTRACT
We explore the emergence of intelligent behavior in artificial systems by inves-
tigating how the complexity of rule-based systems influences the capabilities of
models trained to predict these rules. Our study focuses on elementary cellu-
lar automata (ECA), simple yet powerful one-dimensional systems that generate
behaviors ranging from trivial to highly complex. By training distinct Large Lan-
guage Models (LLMs) on different ECAs, we evaluated the relationship between
the complexity of the data generated by the rules and the models’ ability to learn
effective general representations, as reflected in their performance on downstream
tasks.
Our findings reveal that models trained on more complex data exhibit
greater predictive ability, as demonstrated by their performance on reasoning and
chess move prediction tasks. Both uniform and periodic systems, and often also
highly chaotic systems, resulted in poorer downstream performance, highlighting
a sweet spot of complexity conducive to intelligence. We conjecture that intelli-
gence arises from the ability to predict complexity and that creating intelligence
may require only exposure to complexity.
1
INTRODUCTION
The emergence and nature of intelligence within computational systems have long been subjects of
fascination and rigorous study in the fields of artificial intelligence (AI) and theoretical computation.
Traditional AI methodologies predominantly involve training models on high-quality datasets inher-
ently imbued with human intelligence—such as natural language corpora, expert-annotated datasets,
or data reflecting human cognitive processes (Coleman et al., 2019). This approach operates under
the assumption tha...
Maximum Entropy Reinforcement Learning with Diffusion Policy
Xiaoyi Dong 1 2 Jian Cheng 1 3 4 Xi Sheryl Zhang 1 4
Abstract
The Soft Actor-Critic (SAC) algorithm with a
Gaussian policy has become a mainstream im-
plementation for realizing the Maximum Entropy
Reinforcement Learning (MaxEnt RL) objective,
which incorporates entropy maximization to en-
courage exploration and enhance policy robust-
ness. While the Gaussian policy performs well
on simpler tasks, its exploration capacity and po-
tential performance in complex multi-goal RL
environments are limited by its inherent uni-
modality. In this paper, we employ the diffusion
model, a powerful generative model capable of
capturing complex multimodal distributions, as
the policy representation to fulfill the MaxEnt
RL objective, developing a method named Max-
Ent RL with Diffusion Policy (MaxEntDP). Our
method enables efficient exploration and brings
the policy closer to the optimal MaxEnt pol-
icy. Experimental results on Mujoco benchmarks
show that MaxEntDP outperforms the Gaussian
policy and other generative models within the
MaxEnt RL framework, and performs compara-
bly to other state-of-the-art diffusion-based on-
line RL algorithms.
Our code is available at
https://github.com/diffusionyes/MaxEntDP.
1. Introduction
Reinforcement Learning (RL) has emerged as a powerful
paradigm for training intelligent agents to make decisions in
complex control tasks (Silver et al., 2016; Mnih et al., 2015;
Kaufmann et al., 2023; Kiran et al., 2021; Ibarz et al., 2021).
Traditionally, RL focuses on maximizing the expected cu-
mulative reward, where the agent selects actions that yield
the highest return in each state (Sutton & Barto, 1999). How-
1C2DL, Institute of Automation, Chinese Academy of Sciences
2School of Artificial Intelligence, University of Chinese Academy
of Sciences 3School of Future Technology, University of Chi-
nese Academy of Sciences 4AiRiA. Correspondence to: Xi Sheryl
Zhang <sheryl.zhangxi@gmail.com>.
Pr...
Test-Time Adaptation by Causal Trimming
Yingnan Liu1,2
Rui Qiao1,3
Mong Li Lee1,2
Wynne Hsu1,2
1School of Computing, National University of Singapore
2Institute of Data Science, National University of Singapore
3Singapore-MIT Alliance for Research and Technology
{liu.yingnan, rui.qiao}@u.nus.edu, {dcsleeml, dcshsuw}@nus.edu.sg
Abstract
Test-time adaptation aims to improve model robustness under distribution shifts
by adapting models with access to unlabeled target samples. A primary cause of
performance degradation under such shifts is the model’s reliance on features that
lack a direct causal relationship with the prediction target. We introduce Test-time
Adaptation by Causal Trimming (TACT), a method that identifies and removes
non-causal components from representations for test distributions. TACT applies
data augmentations that preserve causal features while varying non-causal ones. By
analyzing the changes in the representations using Principal Component Analysis,
TACT identifies the highest variance directions associated with non-causal features.
It trims the representations by removing their projections on the identified direc-
tions, and uses the trimmed representations for the predictions. During adaptation,
TACT continuously tracks and refines these directions to get a better estimate of
non-causal features. We theoretically analyze the effectiveness of this approach and
empirically validate TACT on real-world out-of-distribution benchmarks. TACT
consistently outperforms state-of-the-art methods by a significant margin. Our
code is available at https://github.com/NancyQuris/TACT.
1
Introduction
Machine learning models often exhibit significant performance degradation when evaluated on data
drawn from a distribution that differs from their training data distribution [13]. To address this
challenge, test-time adaptation (TTA) has emerged as a promising approach. TTA methods adapt
a pretrained model to the test distribution dynamically, using the incoming tes...
Understanding and Mitigating Memorization in Generative Models via
Sharpness of Probability Landscapes
Dongjae Jeon * 1 Dueun Kim * 2 Albert No 2
Abstract
In this paper, we introduce a geometric framework
to analyze memorization in diffusion models
through the sharpness of the log probability
density. We mathematically justify a previously
proposed score-difference-based memorization
metric by demonstrating its effectiveness in
quantifying sharpness. Additionally, we propose
a novel memorization metric that captures
sharpness at the initial stage of image generation
in latent diffusion models, offering early insights
into potential memorization.
Leveraging this
metric, we develop a mitigation strategy that
optimizes the initial noise of the generation
process using a sharpness-aware regulariza-
tion term.
The code is publicly available at
https://github.com/Dongjae0324/
sharpness_memorization_diffusion.
1. Introduction
Recent advancements in generative models have signifi-
cantly improved data generation across various domains,
including image synthesis (Rombach et al., 2022), natural
language processing (Achiam et al., 2023; Touvron et al.,
2023), and molecular design (Alakhdar et al., 2024). Among
these, diffusion models (Ho et al., 2020; Song et al., 2021c)
have emerged as powerful frameworks, achieving state-of-
the-art results by iteratively refining noisy samples to ap-
proximate complex data distributions (Song et al., 2021b;
Saharia et al., 2022; Rombach et al., 2022).
Despite their successes, diffusion models suffer from mem-
orization, where they replicate training samples instead of
generating novel outputs (Carlini et al., 2023; Somepalli
et al., 2023b; Webster, 2023). This issue is especially con-
*Equal contribution 1Department of Computer Science, Yonsei
University, Seoul, Korea 2Department of Artificial Intelligence,
Yonsei University, Seoul, Korea. Correspondence to: Albert No
<albertno@yonsei.ac.kr>.
Proceedings of the 42 nd International Conferenc...
Aggregation Buffer: Revisiting DropEdge with a New Parameter Block
Dooho Lee 1 Myeong Kong 1 Sagad Hamid 1 2 Cheonwoo Lee 1 Jaemin Yoo 1
Abstract
We revisit DropEdge, a data augmentation tech-
nique for GNNs which randomly removes edges
to expose diverse graph structures during train-
ing. While being a promising approach to ef-
fectively reduce overfitting on specific connec-
tions in the graph, we observe that its potential
performance gain in supervised learning tasks
is significantly limited. To understand why, we
provide a theoretical analysis showing that the
limited performance of DropEdge comes from
the fundamental limitation that exists in many
GNN architectures. Based on this analysis, we
propose Aggregation Buffer, a parameter block
specifically designed to improve the robustness of
GNNs by addressing the limitation of DropEdge.
Our method is compatible with any GNN model,
and shows consistent performance improvements
on multiple datasets. Moreover, our method ef-
fectively addresses well-known problems such as
degree bias or structural disparity as a unifying so-
lution. Code and datasets are available at https:
//github.com/dooho00/agg-buffer.
1. Introduction
Graph-structured data are pervasive across various research
fields and real-world applications, as graphs naturally cap-
ture essential relationships among entities in complex sys-
tems. Graph neural networks (GNNs) have emerged as a
powerful framework to effectively incorporate these rela-
tionships for graph-related tasks. In contrast to traditional
multi-layer perceptrons (MLPs), which solely consider node
features, GNNs additionally take advantage of edge informa-
tion to incorporate crucial interrelations between node fea-
tures (Kipf & Welling, 2017; Gasteiger et al., 2019; Hamil-
ton et al., 2017; Veliˇckovi´c et al., 2018). As a consequence,
1School of Electrical Engineering, KAIST, Daejeon, Re-
public of Korea 2Computer Science Department, University of
M¨unster, M¨unster, Germany. Corresp...
TIMING: Temporality-Aware Integrated Gradients for Time Series Explanation
Hyeongwon Jang * 1 Changhun Kim * 1 2 Eunho Yang 1 2
Abstract
Recent explainable artificial intelligence (XAI)
methods for time series primarily estimate point-
wise attribution magnitudes, while overlooking
the directional impact on predictions, leading
to suboptimal identification of significant points.
Our analysis shows that conventional Integrated
Gradients (IG) effectively capture critical points
with both positive and negative impacts on predic-
tions. However, current evaluation metrics fail
to assess this capability, as they inadvertently
cancel out opposing feature contributions. To ad-
dress this limitation, we propose novel evalua-
tion metrics—Cumulative Prediction Difference
(CPD) and Cumulative Prediction Preservation
(CPP)—to systematically assess whether attribu-
tion methods accurately identify significant pos-
itive and negative points in time series XAI. Un-
der these metrics, conventional IG outperforms
recent counterparts. However, directly applying
IG to time series data may lead to suboptimal
outcomes, as generated paths ignore temporal re-
lationships and introduce out-of-distribution sam-
ples. To overcome these challenges, we introduce
TIMING, which enhances IG by incorporating
temporal awareness while maintaining its theoreti-
cal properties. Extensive experiments on synthetic
and real-world time series benchmarks demon-
strate that TIMING outperforms existing time
series XAI baselines. Our code is available at
https://github.com/drumpt/TIMING.
1. Introduction
Time series data are prevalent across various fields, espe-
cially in safety-critical domains such as healthcare (Sukkar
et al., 2012; Bica et al., 2020; Rangapuram et al., 2021),
energy (Rangapuram et al., 2018; Benidis et al., 2022),
*Equal contribution
1Kim Jaechul Graduate School of AI,
KAIST, Daejeon, South Korea 2AITRICS, Seoul, South Korea.
Correspondence to: Eunho Yang <eunhoy@kaist.ac.kr>.
Proceeding...