GenAI4ELab/papercli-papers huggingface dataset:
Direct Motion Models for Assessing Generated Videos
Authors: Kelsey R Allen; Carl Doersch; Guangyao Zhou; Mohammed Suhail; Danny Driess; Ignacio Rocco; Yulia Rubanova; Thomas Kipf; Mehdi S. M. Sajjadi; Kevin Patrick Murphy; Joao Carreira; Sjoerd van Steenkiste
Venue: ICML 2025 (ICML 2025 poster)

Abstract: A current limitation of video generative video models is that they generate plausible looking frames, but poor motion --- an issue that is not well captured by FVD and other popular methods for evaluating generated videos. Here we go beyond FVD by developing a metric which better measures plausible object interactions and motion. Our novel approach is based on auto-encoding point tracks and yields motion features that can be used to not only compare distributions of videos (as few as one generated and one ground truth, or as many as two datasets), but also for evaluating motion of single videos. We show that using point tracks instead of pixel reconstruction or action recognition features results in a metric which is markedly more sensitive to temporal distortions in synthetic data, and can predict human evaluations of temporal consistency and realism in generated videos obtained from open-source models better than a wide range of alternatives. We also show that by using a point track representation, we can spatiotemporally localize generative video inconsistencies, providing extra interpretability of generated video errors relative to prior work. An overview of the results and link to the code can be found on the project page: trajan-paper.github.io.
Topology-aware Neural Flux Prediction Guided by Physics
Authors: Haoyang Jiang; Jindong Wang; Xingquan Zhu; Yi He
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Graph Neural Networks (GNNs) often struggle in preserving high-frequency components of nodal signals when dealing with directed graphs. Such components are crucial for modeling flow dynamics, without which a traditional GNN tends to treat a graph with forward and reverse topologies equal. To make GNNs sensitive to those high-frequency components thereby being capable to capture detailed topological differences, this paper proposes a novel framework that combines 1) explicit difference matrices that model directional gradients and 2) implicit physical constraints that enforce messages passing within GNNs to be consistent with natural laws. Evaluations on two real-world directed graph data, namely, water flux network and urban traffic flow network, demonstrate the effectiveness of our proposal.
Safety Certificate against Latent Variables with Partially Unidentifiable Dynamics
Authors: Haoming Jing; Yorie Nakahira
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Existing control techniques often assume access to complete dynamics or perfect simulators with fully observable states, which are necessary to verify whether the system remains within a safe set (forward invariance) or safe actions are persistently feasible at all times. However, many systems contain latent variables that make their dynamics partially unidentifiable or cause distribution shifts in the observed statistics between offline and online data, even when the underlying mechanistic dynamics are unchanged. Such “spurious” distribution shifts can break many techniques that use data to learn system models or safety certificates. To address this limitation, we propose a technique for designing probabilistic safety certificates for systems with latent variables. A key technical enabler is the formulation of invariance conditions in probability space, which can be constructed using observed statistics in the presence of distribution shifts due to latent variables. We use this invariance condition to construct a safety certificate that can be implemented efficiently in real-time control. The proposed safety certificate can persistently find feasible actions that control long-term risk to stay within tolerance. Stochastic safe control and (causal) reinforcement learning have been studied in isolation until now. To the best of our knowledge, the proposed work is the first to use causal reinforcement learning to quantify long-term risk for the design of safety certificates. This integration enables safety certificates to efficiently ensure long-term safety in the presence of latent variables. The effectiveness of the proposed safety certificate is demonstrated in numerical simulations.
Zero-shot Meta-learning for Tabular Prediction Tasks with Adversarially Pre-trained Transformer
Authors: Yulun Wu; Doron L Bergman
Venue: ICML 2025 (ICML 2025 poster)

Abstract: We present an Adversarially Pre-trained Transformer (APT) that is able to perform zero-shot meta-learning on tabular prediction tasks without using any real-world dataset to pre-train the model, extending on the recent development of Prior-Data Fitted Networks (PFNs) and TabPFN. Specifically, APT is pre-trained with adversarial synthetic data agents, who continue to shift their underlying data generating distribution and deliberately challenge the model with different synthetic datasets. In addition, we propose a mixture block model architecture that is able to handle classification tasks with arbitrary number of classes, addressing the class size limitation -- a crucial weakness of prior tabular zero-shot learning algorithms. In experiments, we show that our framework matches state-of-the-art performance on small tabular classification tasks without filtering on dataset characteristics such as number of classes and number of missing values, while maintaining an average runtime under one second. On common benchmark dataset suites in both classification and regression, we show that adversarial pre-training was able to enhance TabPFN's performance. In our analysis, we demonstrate that the adversarial synthetic data agents were able to generate a more diverse collection of data compared to the ordinary random generator in TabPFN. In addition, we demonstrate that our mixture block neural design has improved generalizability and greatly accelerated pre-training.
Batch List-Decodable Linear Regression via Higher Moments
Authors: Ilias Diakonikolas; Daniel Kane; Sushrut Karmalkar; Sihan Liu; Thanasis Pittas
Venue: ICML 2025 (ICML 2025 poster)

Abstract: We study the task of list-decodable linear regression using batches, recently introduced by Das et al. 2023.. 
In this setting, we are given $m$ batches with each batch containing 
$n$ points in $\mathbb R^d$. A batch is called clean if the points it contains are i.i.d. samples from an unknown linear regression distribution. 
For a parameter $\alpha \in (0, 1/2)$, an unknown $\alpha$-fraction 
of the batches are clean and no assumptions are made on the remaining batches. 
The goal is to output a small list of vectors at least one of which is close to the true regressor vector in $\ell_2$-norm. Das et al. 2023 gave an efficient algorithm for this task, under natural distributional assumptions, 
with the following guarantee. 
Under the assumption that the batch size satisfies 
$n \geq \tilde{\Omega}(\alpha^{-1})$ and the total number of batches 
is $m = \text{poly}(d, n, 1/\alpha)$, 
their algorithm runs in polynomial time and 
outputs a list of $O(1/\alpha^2)$ vectors at least one of which 
is $\tilde{O}(\alpha^{-1/2}/\sqrt{n})$ close to the target regressor. 
Here we design a new polynomial-time algorithm 
for this task with significantly stronger guarantees under the assumption that the low-degree moments of the covariates distribution are 
Sum-of-Squares (SoS) certifiably bounded.
Specifically, for any constant $\delta>0$, as long as the batch size is 
$n \geq \Omega_{\delta}(\alpha^{-\delta})$
and the degree-$\Theta(1/\delta)$ moments of the covariates are SoS certifiably bounded, 
our algorithm uses $m = \text{poly}((dn)^{1/\delta}, 1/\alpha)$ batches,
runs in polynomial-time, and outputs an $O(1/\alpha)$-sized list of vectors one of which is 
$O(\alpha^{-\delta/2}/\sqrt{n})$ close to the target. That is, our algorithm substantially 
improves both the minimum batch size and the final error guarantee, while achieving the optimal list size. 
Our approach leverages higher-order moment information by carefully combining the SoS paradigm interleaved with an iterative method and a novel list  pruning procedure for this setting. 
In the process, we give an SoS proof of the Marcinkiewicz-Zygmund inequality that may be of broader applicability.
Visual Graph Arena: Evaluating Visual Conceptualization of Vision and Multimodal Large Language Models
Authors: Zahra Babaiee; Peyman Kiasari; Daniela Rus; Radu Grosu
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Recent advancements in multimodal large language models have driven breakthroughs in visual question answering. Yet, a critical gap persists, `conceptualization'—the ability to recognize and reason about the same concept despite variations in visual form, a basic ability of human reasoning. To address this challenge, we introduce the Visual Graph Arena (VGA), a dataset featuring six graph-based tasks designed to evaluate and improve AI systems’ capacity for visual abstraction. VGA uses diverse graph layouts (e.g., Kamada-Kawai vs. planar) to test reasoning independent of visual form. Experiments with state-of-the-art vision models and multimodal LLMs reveal a striking divide: humans achieved near-perfect accuracy across tasks, while models totally failed on isomorphism detection and showed limited success in path/cycle tasks. We further identify behavioral anomalies suggesting pseudo-intelligent pattern matching rather than genuine understanding. These findings underscore fundamental limitations in current AI models for visual understanding. By isolating the challenge of representation-invariant reasoning, the VGA provides a framework to drive progress toward human-like conceptualization in AI visual models. The Visual Graph Arena is available at: \href{https://vga.csail.mit.edu/}{vga.csail.mit.edu}.
From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline
Authors: Tianle Li; Wei-Lin Chiang; Evan Frick; Lisa Dunlap; Tianhao Wu; Banghua Zhu; Joseph E. Gonzalez; Ion Stoica
Venue: ICML 2025 (ICML 2025 poster)

Abstract: The rapid evolution of Large Language Models (LLMs) has outpaced the development of model evaluation, highlighting the need for continuous curation of new, challenging benchmarks. However, manual curation of high-quality, human-aligned benchmarks is expensive and time-consuming. To address this, we introduce BenchBuilder, an automated pipeline that leverages LLMs to curate high-quality, open-ended prompts from large, crowd-sourced datasets, enabling continuous benchmark updates without human in the loop. We apply BenchBuilder to datasets such as Chatbot Arena and WildChat-1M, extracting challenging prompts and utilizing LLM-as-a-Judge for automatic model evaluation. To validate benchmark quality, we propose new metrics to measure a benchmark’s alignment with human preferences and ability to separate models. We release Arena-Hard-Auto, a benchmark consisting 500 challenging prompts curated by BenchBuilder. Arena-Hard-Auto provides 3x higher separation of model performances compared to MT-Bench and achieves 98.6% correlation with human preference rankings, all at a cost of $20. Our work sets a new framework for the scalable curation of automated benchmarks from extensive data.
Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions
Authors: Eray Erturk; Fahad Kamran; Salar Abbaspourazad; Sean Jewell; Harsh Sharma; Yujie Li; Sinead Williamson; Nicholas J Foti; Joseph Futoma
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Wearable devices record physiological and behavioral signals that can improve health predictions. While foundation models are increasingly used for such predictions, they have been primarily applied to low-level sensor data, despite behavioral data often being more informative due to their alignment with physiologically relevant timescales and quantities. We develop foundation models of such behavioral signals using over 2.5B hours of wearable data from 162K individuals, systematically optimizing architectures and tokenization strategies for this unique dataset. Evaluated on 57 health-related tasks, our model shows strong performance across diverse real-world applications including individual-level classification and time-varying health state prediction. The model excels in behavior-driven tasks like sleep prediction, and improves further when combined with representations of raw sensor data. These results underscore the importance of tailoring foundation model design to wearables and demonstrate the potential to enable new health applications.
(How) Do Language Models Track State?
Authors: Belinda Z. Li; Zifan Carl Guo; Jacob Andreas
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Transformer language models (LMs) exhibit behaviors—from storytelling to code generation—that seem to require tracking the unobserved state of an evolving world. How do they do this? We study state tracking in LMs trained or fine-tuned to compose permutations (i.e., to compute the order of a set of objects after a sequence of swaps). Despite the simple algebraic structure of this problem, many other tasks (e.g., simulation of finite automata and evaluation of boolean expressions) can be reduced to permutation composition, making it a natural model for state tracking in general. We show that LMs consistently learn one of two state tracking mechanisms for this task. The first closely resembles the “associative scan” construction used in recent theoretical work by Liu et al. (2023) and Merrill et al. (2024). The second uses an easy-to-compute feature (permutation parity) to partially prune the space of outputs, and then refines this with an associative scan. LMs that learn the former algorithm tend to generalize better and converge faster, and we show how to steer LMs toward one or the other with intermediate training tasks that encourage or suppress the heuristics. Our results demonstrate that transformer LMs, whether pre-trained or fine-tuned, can learn to implement efficient and interpretable state-tracking mechanisms, and the emergence of these mechanisms can be predicted and controlled. Code and data are available at https://github.com/belindal/state-tracking
In-Context Fine-Tuning for Time-Series Foundation Models
Authors: Matthew Faw; Rajat Sen; Yichen Zhou; Abhimanyu Das
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Motivated by the recent success of time-series foundation models for zero-shot forecasting, we present a methodology for _in-context fine-tuning_ of a time-series foundation model. In particular, we design a pretrained foundation model that can be prompted (at inference time) with multiple time-series examples, in order to forecast a target time-series into the future. Our foundation model is specifically trained to utilize examples from multiple related time-series in its context window (in addition to the history of the target time-series) to help it adapt to the specific distribution of the target domain at inference time.  We show that such a foundation model that uses in-context examples at inference time can obtain much better performance on popular forecasting benchmarks compared to supervised deep learning methods, statistical models, and other time series foundation models.  Interestingly, our in-context fine-tuning approach even matches the performance of a foundation model that is explicitly fine-tuned on the target domain.
Understanding Fixed Predictions via Confined Regions
Authors: Connor Lawless; Tsui-Wei Weng; Berk Ustun; Madeleine Udell
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Machine learning models can assign fixed predictions that preclude individuals from changing their outcome. Existing approaches to audit fixed predictions do so on a pointwise basis, which requires access to an existing dataset of individuals and may fail to anticipate fixed predictions in out-of-sample data. This work presents a new paradigm to identify fixed predictions by finding confined regions of the feature space in which all individuals receive fixed predictions. This paradigm enables the certification of recourse for out-of-sample data, works in settings without representative datasets, and provides interpretable descriptions of individuals with fixed predictions. We develop a fast method to discover confined regions for linear classifiers using mixed-integer quadratically constrained programming. We conduct a comprehensive empirical study of confined regions across diverse applications. Our results highlight that existing pointwise verification methods fail to anticipate future individuals with fixed predictions, while our method both identifies them and provides an interpretable description.
Principled Data Selection for Alignment: The Hidden Risks of Difficult Examples
Authors: Chengqian Gao; Haonan Li; Liu Liu; Zeke Xie; Peilin Zhao; zhiqiang xu
Venue: ICML 2025 (ICML 2025 poster)

Abstract: The alignment of large language models (LLMs) often assumes that using more clean data yields better outcomes, overlooking the match between model capacity and example difficulty. Challenging this, we propose a new principle: *Preference data vary in difficulty, and overly difficult examples hinder alignment, by exceeding the model's capacity*. Through systematic experimentation, we validate this principle with three key findings: (1) preference examples vary in difficulty, as evidenced by consistent learning orders across alignment runs; (2) overly difficult examples significantly degrade performance across four LLMs and two datasets; and (3) the capacity of a model dictates its threshold for handling difficult examples, underscoring a critical relationship between data selection and model capacity. Building on this principle, we introduce *Selective DPO*, which filters out overly difficult examples. This simple adjustment improves alignment performance by 9-16\% in win rates on the AlpacaEval 2 benchmark compared to the DPO baseline, surpassing a series of DPO variants with different algorithmic adjustments. These results together illuminate the importance of aligning data difficulty with model capacity, offering a transformative perspective for improving alignment strategies in LLMs. Code is available at https://github.com/glorgao/SelectiveDPO
Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models
Authors: Konstantin Donhauser; Kristina Ulicna; Gemma Elyse Moran; Aditya Ravuri; Kian Kenyon-Dean; Cian Eastwood; Jason Hartford
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Sparse dictionary learning (DL) has emerged as a powerful approach to extract semantically meaningful concepts from the internals of large language models (LLMs) trained mainly in the text domain. In this work, we explore whether DL can extract meaningful concepts from less human-interpretable scientific data, such as vision foundation models trained on cell microscopy images, where limited prior knowledge exists about which high-level concepts should arise. We propose a novel combination of a sparse DL algorithm, Iterative Codebook Feature Learning (ICFL), with a PCA whitening pre-processing step derived from control data. Using this combined approach, we successfully retrieve biologically meaningful concepts, such as cell types and genetic perturbations. Moreover, we demonstrate how our method reveals subtle morphological changes arising from human-interpretable interventions, offering a promising new direction for scientific discovery via mechanistic interpretability in bioimaging.
Sliding Puzzles Gym: A Scalable Benchmark for State Representation in Visual Reinforcement Learning
Authors: Bryan Lincoln Marques de Oliveira; Luana Guedes Barros Martins; Bruno Brandão; Murilo Lopes da Luz; telma woerle de lima soares; Luckeciano Carvalho Melo
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Effective visual representation learning is crucial for reinforcement learning (RL) agents to extract task-relevant information from raw sensory inputs and generalize across diverse environments. However, existing RL benchmarks lack the ability to systematically evaluate representation learning capabilities in isolation from other learning challenges. To address this gap, we introduce the Sliding Puzzles Gym (SPGym), a novel benchmark that transforms the classic 8-tile puzzle into a visual RL task with images drawn from arbitrarily large datasets. SPGym's key innovation lies in its ability to precisely control representation learning complexity through adjustable grid sizes and image pools, while maintaining fixed environment dynamics, observation, and action spaces. This design enables researchers to isolate and scale the visual representation challenge independently of other learning components. Through extensive experiments with model-free and model-based RL algorithms, we uncover fundamental limitations in current methods' ability to handle visual diversity. As we increase the pool of possible images, all algorithms exhibit in- and out-of-distribution performance degradation, with sophisticated representation learning techniques often underperforming simpler approaches like data augmentation. These findings highlight critical gaps in visual representation learning for RL and establish SPGym as a valuable tool for driving progress in robust, generalizable decision-making systems.
Causal Abstraction Learning based on the Semantic Embedding Principle
Authors: Gabriele D'Acunto; Fabio Massimo Zennaro; Yorgos Felekis; Paolo Di Lorenzo
Venue: ICML 2025 (ICML 2025 poster)

Abstract: Structural causal models (SCMs) allow us to investigate complex systems at multiple levels of resolution.
The causal abstraction (CA) framework formalizes the mapping between high- and low-level SCMs. 
We address CA learning in a challenging and realistic setting, where SCMs are inaccessible, interventional data is unavailable, and sample data is misaligned.
A key principle of our framework is *semantic embedding*, formalized as the high-level distribution lying on a subspace of the low-level one. 
This principle naturally links linear CA to the geometry of the *Stiefel manifold*.
We present a category-theoretic approach to SCMs that enables the learning of a CA by finding a morphism between the low- and high-level probability measures, adhering to the semantic embedding principle.
Consequently, we formulate a general CA learning problem.
As an application, we solve the latter problem for linear CA; considering Gaussian measures and the Kullback-Leibler divergence as an objective.
Given the nonconvexity of the learning task, we develop three algorithms building upon existing paradigms for Riemannian optimization.
We demonstrate that the proposed methods succeed on both synthetic and real-world brain data with different degrees of prior information about the structure of CA.