AIS-Bench/AIS-Bench huggingface dataset:
Constrained Online Convex Optimization with Polyak Feasibility Steps
Spencer Hutchinson 1 Mahnoosh Alizadeh 1
Abstract
In this work, we study online convex optimiza-
tion with a fixed constraint function g : Rd →
R.
Prior work on this problem has shown
O(
√
T) regret and cumulative constraint satisfac-
tion PT
t=1 g(xt) ≤0, while only accessing the
constraint value and subgradient at the played ac-
tions g(xt), ∂g(xt). Using the same constraint
information, we show a stronger guarantee of any-
time constraint satisfaction g(xt) ≤0 ∀t ∈[T],
and matching O(
√
T) regret guarantees. These
contributions are thanks to our approach of using
Polyak feasibility steps to ensure constraint sat-
isfaction, without sacrificing regret. Specifically,
after each step of online gradient descent, our
algorithm applies a subgradient descent step on
the constraint function where the step-size is cho-
sen according to the celebrated Polyak step-size.
We further validate this approach with numerical
experiments.
1. Introduction
We study the problem of online convex optimization (OCO)
where, in each round t = 1, 2, ..., T, a player chooses an ac-
tion xt from an action set X ⊆Rd and then suffers the cost
ft(xt) according to an adversarially-chosen convex function
ft (Zinkevich, 2003). The player’s goal is to minimize the
regret with respect to the best single action in hindsight,
RegT :=
T
X
t=1
ft(xt) −min
x∈X
T
X
t=1
ft(x).
The OCO problem has emerged as a fundamental setting
for various machine learning domains, such as stochastic
optimization (Cesa-Bianchi et al., 2004), non-convex opti-
1Department of Electrical and Computer Engineering, Uni-
versity of California-Santa Barbara, Santa Barbara, Califor-
nia, USA. Correspondence to: Spencer Hutchinson <shutchin-
son@ucsb.edu>, Mahnoosh Alizadeh <alizadeh@ucsb.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
mization (Cutkosky et al., 2023), ...
You Always Recognize Me (YARM): Robust Texture Synthesis
Against Multi-View Corruption
Weihang Ran 1 Wei Yuan 2 Yinqiang Zheng 1
Abstract
Damage to imaging systems and complex external
environments often introduce corruptions, which
can impair the performance of deep learning mod-
els pretrained on high-quality image data. Previ-
ous methods have focused on restoring degraded
images or fine-tuning models to adapt to out-
of-distribution data. However, these approaches
struggle with complex, unknown corruptions and
often reduce model accuracy on high-quality data.
Inspired by the use of warning colors and camou-
flage in the real world, we propose designing a
robust appearance that can enhance model recog-
nition of low-quality image data. Furthermore,
we demonstrate that certain universal features in
radiance fields can be applied across objects of the
same class with different geometries. We also ex-
amine the impact of different proxy models on the
transferability of robust appearances. Extensive
experiments demonstrate the effectiveness of our
proposed method, which outperforms existing im-
age restoration and model fine-tuning approaches
across different experimental settings, and retains
effectiveness when transferred to models with dif-
ferent architectures. Code will be available at
https://github.com/SilverRAN/YARM.
1. Introduction
Neural network-based deep learning technologies have
made significant impacts across various domains in mod-
ern society, including facial recognition, autonomous driv-
ing, and 3D reconstruction. However, previous research
has shown that neural network models can yield erroneous
predictions in certain scenarios, such as under adversarial
attacks or image degradation (Hosseini et al., 2017; Dodge
1The University of Tokyo, Japan
2Tohoku University,
Japan. Correspondence to: Yinqiang Zheng <yqzheng@ai.u-
tokyo.ac.jp>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright ...
Time Series Representations with Hard-Coded Invariances
Thibaut Germain * 1 Chrysoula Kosma * 1 Laurent Oudre 1
Abstract
Automatically extracting robust representations
from large and complex time series data is becom-
ing imperative for several real-world applications.
Unfortunately, the potential of common neural
network architectures in capturing invariant prop-
erties of time series remains relatively underex-
plored. For instance, convolutional layers often
fail to capture underlying patterns in time series
inputs that encompass strong deformations, such
as trends. Indeed, invariances to some deforma-
tions may be critical for solving complex time se-
ries tasks, such as classification, while guarantee-
ing good generalization performance. To address
these challenges, we mathematically formulate
and technically design efficient and hard-coded
invariant convolutions for specific group actions
applicable to the case of time series. We construct
these convolutions by considering specific sets of
deformations commonly observed in time series,
including scaling, offset shift, and trend. We fur-
ther combine the proposed invariant convolutions
with standard convolutions in single embedding
layers, and we showcase the layer’s capacity to
capture complex invariant time series properties
in several scenarios.
1. Introduction
Recently, there has been a growing interest in applying
machine learning to time series data, further necessitated by
the challenging properties of time series, including varying
modalities, high noise levels, and distribution shifts.
Machine learning for time series has gradually moved from
statistical methods, e. g., autoregressive models (Box et al.,
*Equal contribution 1Universit´e Paris-Saclay, Universit´e Paris
Cit´e, ENS Paris Saclay, CNRS, SSA, INSERM, Centre Borelli,
F-91190, Gif-sur-Yvette, France. Correspondence to: Thibaut Ger-
main <thibaut.germain@ens-paris-saclay.fr>, Chrysoula Kosma
<chrysoula.kosma@ens-paris-saclay.fr>.
Proceedings ...
Influence Guided Context Selection for Effective
Retrieval-Augmented Generation
Jiale Deng, Yanyan Shen∗, Ziyuan Pei, Youmin Chen, Linpeng Huang
Shanghai Jiao Tong University
{jialedeng, shenyy, pzy_live, chenyoumin, lphuang}@sjtu.edu.cn
Abstract
Retrieval-Augmented Generation (RAG) addresses large language model (LLM)
hallucinations by grounding responses in external knowledge, but its effectiveness
is compromised by poor-quality retrieved contexts containing irrelevant or noisy
information. While existing approaches attempt to improve performance through
context selection based on predefined context quality assessment metrics, they
show limited gains over standard RAG. We attribute this limitation to their failure
in holistically utilizing available information (query, context list, and generator)
for comprehensive quality assessment. Inspired by recent advances in data se-
lection, we reconceptualize context quality assessment as an inference-time data
valuation problem and introduce the Contextual Influence Value (CI value). This
novel metric quantifies context quality by measuring the performance degradation
when removing each context from the list, effectively integrating query-aware rele-
vance, list-aware uniqueness, and generator-aware alignment. Moreover, CI value
eliminates complex selection hyperparameter tuning by simply retaining contexts
with positive CI values. To address practical challenges of label dependency and
computational overhead, we develop a parameterized surrogate model for CI value
prediction during inference. The model employs a hierarchical architecture that
captures both local query-context relevance and global inter-context interactions,
trained through oracle CI value supervision and end-to-end generator feedback.
Extensive experiments across 8 NLP tasks and multiple LLMs demonstrate that
our context selection method significantly outperforms state-of-the-art baselines,
effectively filtering poor-quality contexts while preserving cr...
Implicit Degree Bias in the Link Prediction Task
Rachith Aiyappa 1 Xin Wang 2 Munjung Kim 1 Ozgur Can Seckin 1 Yong-Yeol Ahn 1 Sadamori Kojaku 2
Abstract
Link prediction—the task of distinguishing ac-
tual hidden edges from random unconnected node
pairs—is a quintessential task in graph machine
learning. Despite being widely accepted as a uni-
versal benchmark and a downstream task for rep-
resentation learning, its validity is seldom ques-
tioned. Here, we show that the common edge
sampling procedure in link prediction introduces
an implicit bias toward high-degree nodes and
produces a skewed evaluation that favors methods
overly reliant on node degree, to the extent that
a “null” method based solely on node degree can
nearly match optimal performance. To address
this, we propose a degree-corrected link predic-
tion task that offers a more accurate assessment
that aligns better with performance in recommen-
dation tasks. Finally, we demonstrate that this
degree-corrected benchmark can more effectively
train graph machine-learning models by reduc-
ing overfitting to node degrees and facilitating the
learning of relevant structures in graphs.
1. Introduction
Standardized benchmarks like ImageNet (Deng et al., 2009;
Krizhevsky et al., 2012) and SQuAD (Rajpurkar et al., 2016;
2018) play a pivotal role in driving progress in machine
learning by fostering competition through setting clear, mea-
surable goals. In graph machine learning, a core benchmark
is link prediction, which involves identifying missing edges
in a graph, with diverse applications including the recom-
mendations of friends and contents (Huang et al., 2005;
Kunegis & Lommatzsch, 2009; Menon & Elkan, 2011;
Wang et al., 2014), knowledge discoveries (Bordes et al.,
*Equal contribution
1Center for Complex Networks and Sys-
tems Research, Luddy School of Informatics, Computing, and
Engineering, Indiana University, Bloomington, IN, USA 2School
of Systems Science and Industrial Engineering, Binghamton Uni-
ver...
Zero-Shot Generalization of GNNs over Distinct Attribute Domains
Yangyi Shen 1 Jincheng Zhou 2 Beatrice Bevilacqua 2 Joshua Robinson 1 Charilaos Kanatsoulis 1
Jure Leskovec 1 Bruno Ribeiro 2
Abstract
Traditional Graph Neural Networks (GNNs) can-
not generalize to new graphs with node attributes
different from the training ones, making zero-shot
generalization across different node attribute do-
mains an open challenge in graph machine learn-
ing. In this paper, we propose STAGE, which en-
codes statistical dependencies between attributes
rather than individual attribute values, which may
differ in test graphs. By assuming these depen-
dencies remain invariant under changes in node
attributes, STAGE achieves provable generaliza-
tion guarantees for a family of domain shifts. Em-
pirically, STAGE demonstrates strong zero-shot
performance on medium-sized datasets: when
trained on multiple graph datasets with differ-
ent attribute spaces (varying in types and num-
ber) and evaluated on graphs with entirely new
attributes, STAGE achieves a relative improve-
ment in Hits@1 between 40% to 103% in link
prediction and a 10% improvement in node clas-
sification compared to state-of-the-art baselines.
1. Introduction
Zero-shot generalization refers to the ability of the model to
handle unseen test data without additional training or adap-
tation (Larochelle et al., 2008; Xian et al., 2017; Wang et al.,
2022). An essential prerequisite for zero-shot generalization
is a unified input space where models can learn and transfer
prediction patterns across domains. While this challenge
has been addressed in areas like natural language through
tokenization techniques that represent any text through a
fixed vocabulary (Samuel & Øvrelid, 2023), graphs present
unique challenges in achieving such unified input space.
Attributes in graphs can vary significantly across domains.
1Department of Computer Science, Stanford University, Stan-
ford, USA 2Department of Computer Science, Purdue Univ...
Published as a conference paper at ICLR 2025
FACILITATING MULTI-TURN FUNCTION CALLING FOR
LLMS VIA COMPOSITIONAL INSTRUCTION TUNING
Mingyang Chen1∗, Haoze Sun1∗, Tianpeng Li1∗, Fan Yang1, Hao Liang2, Keer Lu2,
Bin Cui2, Wentao Zhang2†, Zenan Zhou1†, Weipeng Chen1
1Baichuan Inc., 2 Peking University
{chenmingyang, sunhaoze, litianpeng, zhouzenan}@baichuan-inc.com
wentao.zhang@pku.edu.cn
ABSTRACT
Large Language Models (LLMs) have exhibited significant potential in perform-
ing diverse tasks, including the ability to call functions or use external tools to
enhance their performance. While current research on function calling by LLMs
primarily focuses on single-turn interactions, this paper addresses the overlooked
necessity for LLMs to engage in multi-turn function calling—critical for handling
compositional, real-world queries that require planning with functions but not only
use functions. To facilitate this, we introduce an approach, BUTTON, which gen-
erates synthetic compositional instruction tuning data via bottom-up instruction
construction and top-down trajectory generation. In the bottom-up phase, we gen-
erate simple atomic tasks based on real-world scenarios and build compositional
tasks using heuristic strategies based on atomic tasks. Corresponding function def-
initions are then synthesized for these compositional tasks. The top-down phase
features a multi-agent environment where interactions among simulated humans,
assistants, and tools are utilized to gather multi-turn function calling trajectories.
This approach ensures task compositionality and allows for effective function and
trajectory generation by examining atomic tasks within compositional tasks. We
produce a dataset BUTTONInstruct comprising 8k data points and demonstrate
its effectiveness through extensive experiments across various LLMs 1.
1
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable capabilities across various
tasks (OpenAI, 2023; Dubey et al., 2024). Beyond gener...
VCT: Training Consistency Models with Variational Noise Coupling
Gianluigi Silvestri∗1 2 Luca Ambrogioni 2 Chieh-Hsin Lai 3 Yuhta Takida 3 Yuki Mitsufuji 3 4
Abstract
Consistency Training (CT) has recently emerged
as a strong alternative to diffusion models for im-
age generation.
However, non-distillation CT
often suffers from high variance and instabil-
ity, motivating ongoing research into its training
dynamics. We propose Variational Consistency
Training (VCT), a flexible and effective frame-
work compatible with various forward kernels,
including those in flow matching. Its key inno-
vation is a learned noise-data coupling scheme
inspired by Variational Autoencoders, where a
data-dependent encoder models noise emission.
This enables VCT to adaptively learn noise-to-
data pairings, reducing training variance relative
to the fixed, unsorted pairings in classical CT. Ex-
periments on multiple image datasets demonstrate
significant improvements: our method surpasses
baselines, achieves state-of-the-art FID among
non-distillation CT approaches on CIFAR-10, and
matches SoTA performance on ImageNet 64 × 64
with only two sampling steps. Code is available
at https://github.com/sony/vct.
1. Introduction
Generative Models are deep learning algorithms designed
to learn the underlying probability distribution of a given
dataset, in order to then generate samples coming from such
a distribution. Some widely used models are Generative
Adversarial Networks (Goodfellow et al., 2014), Variational
Autoencoders (VAE; Kingma & Welling, 2013; Rezende
et al., 2014), and Normalizing Flows (Chen et al., 2018;
Papamakarios et al., 2021; Kobyzev et al., 2020). More
∗Work done during an internship at Sony AI 1OnePlanet Re-
search Center, imec-the Netherlands, Wageningen, the Nether-
lands 2Donders Institute for Brain, Cognition and Behaviou, Ni-
jmegen, the Netherlands 3Sony AI, Tokyo, Japan 4Sony Group
Corporation.
Correspondence to: Gianluigi Silvestri <gian-
luigi.silvestri@imec.nl, gi...
Steerable Transformers for Volumetric Data
Soumyabrata Kundu 1 Risi Kondor 1 2
Abstract
We introduce Steerable Transformers, an exten-
sion of the Vision Transformer that is equivari-
ant to the action of the Special Euclidean group
SE(d). We propose an steerable self-attention
mechanism that operates on features extracted
by steerable convolutions. Our experiments in
both two and three dimensions show augmenting
steerable convolutional networks with steerable
transformer leads to improved performance.
1. Introduction
Transformers have emerged as the preferred architecture
for natural language processing tasks, with recent powerful
models like Chat-GPT employing this framework. Their
relatively straightforward design, coupled with remarkable
success, has led to their widespread adoption across various
domains, including image classification (Dosovitskiy et al.,
2021), object detection (Carion et al., 2020), and graph
based problems (Dwivedi & Bresson, 2020).
The self-
attention mechanism (Bahdanau et al., 2014) employed in
transformer architectures has proven to be crucial for cap-
turing relationships between different parts of the input se-
quence. Dosovitskiy et al. (2021) introduced transformers
as an alternative to traditional convolutional architectures
for vision tasks.
Unlike convolutional neural networks
(CNNs), which focus on local neighborhoods, transform-
ers excel at capturing relations across different parts of the
input.
Equivariant neural network architectures have gained sig-
nificant popularity in recent years due to their inherent abil-
ity to comprehend the underlying symmetries of problems,
making them highly effective tools for real world tasks. For
instance, achieving equivariance to the permutation group
Sn is crucial in graph based problems, since the structure
1Department of Statistics, University of Chicago, Chicago,
USA 2Department of Computer Science, University of Chicago,
Chicago,
USA. Correspondence to:
Soumyabrata Kundu
<soumyabratak...
CoastalBench: A Decade-Long High-Resolution Dataset to Emulate Complex
Coastal Processes
Zelin Xu 1 Yupu Zhang 1 Tingsong Xiao 1 Maitane Olabarrieta 2 Jose M. Gonzalez-Ondina 2 Zibo Liu 1
Shigang Chen 1 Zhe Jiang 1
Abstract
Over 40% of the global population lives within
100 kilometers of the coast, which contributes
more than $8 trillion annually to the global
economy. Unfortunately, coastal ecosystems are
increasingly vulnerable to more frequent and
intense extreme weather events and rising sea
levels. Coastal scientists use numerical models to
simulate complex physical processes, but these
models are often slow and expensive. In recent
years, deep learning has become a promising
alternative to reduce the cost of numerical models.
However, progress has been hindered by the
lack of a large-scale, high-resolution coastal
simulation dataset to train and validate deep
learning models. Existing studies often focus on
relatively small datasets and simple processes. To
fill this gap, we introduce a decade-long, high-
resolution (<100m) coastal circulation modeling
dataset on a real-world 3D mesh in southwest
Florida with around 6 million cells. The dataset
contains
key
oceanography
variables
(e.g.,
current velocities, free surface level, temperature,
salinity) alongside external atmospheric and river
forcings.
We evaluated a customized Vision
Transformer model that takes initial and boundary
conditions and external forcings and predicts
ocean variables at varying lead times. The dataset
provides an opportunity to benchmark novel
deep learning models for high-resolution coastal
simulations (e.g., physics-informed machine
learning, neural operator learning). The code and
dataset can be accessed at https://github.
com/spatialdatasciencegroup/
CoastalBench.
1Department of Computer & Information Science & Engineer-
ing, University of Florida, Gainesville, FL, USA 2Department of
Civil & Coastal Engineering, University of Florida, Gainesville,
FL, USA. Correspondence to: Zhe Jia...
Published as a conference paper at ICLR 2025
PRIVACY-AWARE LIFELONG LEARNING
Ozan ¨Ozdenizci1, Elmar Rueckert1, Robert Legenstein2
1 Chair of Cyber-Physical-Systems, Montanuniversit¨at Leoben, Austria
2 Institute of Machine Learning and Neural Computation, Graz University of Technology, Austria
{ozan.oezdenizci,elmar.rueckert}@unileoben.ac.at
robert.legenstein@tugraz.at
ABSTRACT
Lifelong learning algorithms enable models to incrementally acquire new knowl-
edge without forgetting previously learned information. Contrarily, the field of
machine unlearning focuses on explicitly forgetting certain previous knowledge
from pretrained models when requested, in order to comply with data privacy reg-
ulations on the right-to-be-forgotten. Enabling efficient lifelong learning with the
capability to selectively unlearn sensitive information from models presents a crit-
ical and largely unaddressed challenge with contradicting objectives. We address
this problem from the perspective of simultaneously preventing catastrophic for-
getting and allowing forward knowledge transfer during task-incremental learn-
ing, while ensuring exact task unlearning and minimizing memory requirements,
based on a single neural network model to be adapted. Our proposed solution,
privacy-aware lifelong learning (PALL), involves optimization of task-specific
sparse subnetworks with parameter sharing within a single architecture. We ad-
ditionally utilize an episodic memory rehearsal mechanism to facilitate exact un-
learning without performance degradations. We empirically demonstrate the scal-
ability of PALL across various architectures in image classification, and provide
a state-of-the-art solution that uniquely integrates lifelong learning and privacy-
aware unlearning mechanisms for responsible AI applications.
1
INTRODUCTION
Lifelong learning algorithms enhance the ability of machine learning models to incrementally ac-
quire new skills or integrate new knowledge over time from sequentially o...
Shifting Time: Time-series Forecasting with Khatri-Rao Neural Operators
Srinath Dama 1 Kevin Course 1 Prasanth B. Nair 1
Abstract
We present an operator-theoretic framework for
temporal and spatio-temporal forecasting based
on learning a continuous time-shift operator. Our
operator learning paradigm offers a continuous
relaxation of the discrete lag factor used in tradi-
tional autoregressive models, enabling the history
of a system up to a given time to be mapped to its
future values. We parametrize the time-shift oper-
ator using Khatri-Rao neural operators (KRNOs),
a novel architecture based on non-stationary inte-
gral transforms with nearly linear computational
scaling. Our framework naturally handles irregu-
larly sampled observations and enables forecast-
ing at super-resolution in both space and time.
Extensive numerical studies across diverse tempo-
ral and spatio-temporal benchmarks demonstrate
that our approach achieves state-of-the-art or com-
petitive performance with leading methods.
1. Introduction
Time series forecasting is a fundamental problem in ma-
chine learning and statistics with applications to a broad
spectrum of problems encountered in all branches of sci-
ence, engineering, and finance (Roberts et al., 2013; Milani
et al., 2017; Siami-Namini & Namin, 2018). At a high-level,
time-series problems are concerned with forecasting the fu-
ture values of quantities of interest given past observations
of the same or correlated quantities.
The majority of methods for time-series forecasting largely
fall into the categories of autoregressive moving average
models and their variants (Box & Jenkins, 1976; Girard,
2004), and deep autoregressive models with memory (El-
man, 1990; Hochreiter & Schmidhuber, 1997; Salinas et al.,
2020). With the tremendous success of transformer-based
1Institute
for
Aerospace
Studies,
University
of
Toronto,
ON,
Canada.
Correspondence
to:
Srinath
Dama
<srinath.dama@mail.utoronto.ca>,
Kevin
Course
<kevin.course@mail.utoronto...