GenAI4ELab/papercli-papers huggingface dataset:
Discriminative Calibration: Check Bayesian Computation from Simulations and Flexible Classifier
Authors: Yuling Yao; Justin Domke
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: To check the accuracy of Bayesian computations, it is common to use rank-based simulation-based calibration (SBC). However, SBC has drawbacks: The test statistic is somewhat ad-hoc, interactions are difficult to examine, multiple testing is a challenge, and the resulting p-value is not a divergence metric. We propose to replace the marginal rank test with a flexible classification approach that learns test statistics from data. This measure typically has a higher statistical power than the SBC test and returns an interpretable divergence measure of miscalibration, computed from classification accuracy. This approach can be used with different data generating processes to address simulation-based inference or traditional inference methods like Markov chain Monte Carlo or variational inference. We illustrate an automated implementation using neural networks and statistically-inspired features, and validate the method with numerical and real data experiments.
Online POMDP Planning with Anytime Deterministic Guarantees
Authors: Moran Barenboim; Vadim Indelman
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: Autonomous agents operating in real-world scenarios frequently encounter uncertainty and make decisions based on incomplete information. Planning under uncertainty can be mathematically formalized using partially observable Markov decision processes (POMDPs). However, finding an optimal plan for POMDPs can be computationally expensive and is feasible only for small tasks. In recent years, approximate algorithms, such as tree search and sample-based methodologies, have emerged as state-of-the-art POMDP solvers for larger problems. Despite their effectiveness, these algorithms offer only probabilistic and often asymptotic guarantees toward the optimal solution due to their dependence on sampling. To address these limitations, we derive a deterministic relationship between a simplified solution that is
easier to obtain and the theoretically optimal one. First, we derive bounds for selecting a subset of the observations to branch from while computing a complete belief at each posterior node. Then, since a complete belief update may be computationally demanding, we extend the bounds to support reduction of both the state and the observation spaces. We demonstrate how our guarantees can be integrated with existing state-of-the-art solvers that sample a subset of states and observations. As a result, the returned solution holds deterministic bounds relative to the optimal policy. Lastly, we substantiate our findings with supporting experimental results.
Prediction and Control in Continual Reinforcement Learning
Authors: Nishanth Anand; Doina Precup
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: Temporal difference (TD) learning is often used to update the estimate of the value function which is used by RL agents to extract useful policies. In this paper, we focus on value function estimation in continual reinforcement learning. We propose to decompose the value function into two components which update at different timescales: a _permanent_ value function, which holds general knowledge that persists over time, and a _transient_ value function, which allows quick adaptation to new situations. We establish theoretical results showing that our approach is well suited for continual learning and draw connections to the complementary learning systems (CLS) theory from neuroscience. Empirically, this approach improves performance significantly on both prediction and control problems.
Residual Q-Learning: Offline and Online Policy Customization without Value
Authors: Chenran Li; Chen Tang; Haruki Nishimura; Jean Mercat; Masayoshi Tomizuka; Wei Zhan
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: Imitation Learning (IL) is a widely used framework for learning imitative behavior from demonstrations. It is especially appealing for solving complex real-world tasks where handcrafting reward function is difficult, or when the goal is to mimic human expert behavior. However, the learned imitative policy can only follow the behavior in the demonstration. When applying the imitative policy, we may need to customize the policy behavior to meet different requirements coming from diverse downstream tasks. Meanwhile, we still want the customized policy to maintain its imitative nature. To this end, we formulate a new problem setting called policy customization. It defines the learning task as training a policy that inherits the characteristics of the prior policy while satisfying some additional requirements imposed by a target downstream task. We propose a novel and principled approach to interpret and determine the trade-off between the two task objectives. Specifically, we formulate the customization problem as a Markov Decision Process (MDP) with a reward function that combines 1) the inherent reward of the demonstration; and 2) the add-on reward specified by the downstream task. We propose a novel framework, Residual Q-learning, which can solve the formulated MDP by leveraging the prior policy without knowing the inherent reward or value function of the prior policy. We derive a family of residual Q-learning algorithms that can realize offline and online policy customization, and show that the proposed algorithms can effectively accomplish policy customization tasks in various environments. Demo videos and code are available on our website: https://sites.google.com/view/residualq-learning.
Near Optimal Reconstruction of Spherical Harmonic Expansions
Authors: Amir Zandieh; Insu Han; Haim Avron
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We propose an algorithm for robust recovery of the spherical harmonic expansion of functions defined on the $d$-dimensional unit sphere $\mathbb{S}^{d-1}$ using a near-optimal number of function evaluations. We show that for any $f\in L^2(\mathbb{S}^{d-1})$, the number of evaluations of $f$ needed to recover its degree-$q$ spherical harmonic expansion equals the dimension of the space of spherical harmonics of degree at most $q$, up to a logarithmic factor. Moreover, we develop a simple yet efficient kernel regression-based algorithm to recover degree-$q$ expansion of $f$ by only evaluating the function on uniformly sampled points on $\mathbb{S}^{d-1}$. Our algorithm is built upon the connections between spherical harmonics and Gegenbauer polynomials. Unlike the prior results on fast spherical harmonic transform, our proposed algorithm works efficiently using a nearly optimal number of samples in any dimension $d$. Furthermore, we illustrate the empirical performance of our algorithm on numerical examples.
Automated Classification of Model Errors on ImageNet
Authors: Momchil Peychev; Mark Niklas Mueller; Marc Fischer; Martin Vechev
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: While the ImageNet dataset has been driving computer vision research over the past decade, significant label noise and ambiguity have made top-1 accuracy an insufficient measure of further progress. To address this, new label-sets and evaluation protocols have been proposed for ImageNet showing that state-of-the-art models already achieve over 95% accuracy and shifting the focus on investigating why the remaining errors persist.

Recent work in this direction employed a panel of experts to manually categorize all remaining classification errors for two selected models. However, this process is time-consuming, prone to inconsistencies, and requires trained experts, making it unsuitable for regular model evaluation thus limiting its utility. To overcome these limitations, we propose the first automated error classification framework, a valuable tool to study how modeling choices affect error distributions. We use our framework to comprehensively evaluate the error distribution of over 900 models. Perhaps surprisingly, we find that across model architectures, scales, and pre-training corpora, top-1 accuracy is a strong predictor for the *portion* of all error types. In particular, we observe that the portion of severe errors drops significantly with top-1 accuracy indicating that, while it underreports a model's true performance, it remains a valuable performance metric.

We release all our code at https://github.com/eth-sri/automated-error-analysis.
Towards robust and generalizable representations of extracellular data using contrastive learning
Authors: Ankit Vishnubhotla; Charlotte Loh; Akash Srivastava; Liam Paninski; Cole Lincoln Hurwitz
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: Contrastive learning is quickly becoming an essential tool in neuroscience for extracting robust and meaningful representations of neural activity. Despite numerous applications to neuronal population data, there has been little exploration of how these methods can be adapted to key primary data analysis tasks such as spike sorting or cell-type classification. In this work, we propose a novel contrastive learning framework, CEED (Contrastive Embeddings for Extracellular Data), for high-density extracellular recordings. We demonstrate that through careful design of the network architecture and data augmentations, it is possible to generically extract representations that far outperform current specialized approaches. We validate our method across multiple high-density extracellular recordings. All code used to run CEED can be found at https://github.com/ankitvishnu23/CEED.
The Gain from Ordering in Online Learning
Authors: Vasilis Kontonis; Mingchen Ma; Christos Tzamos
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We study fixed-design online learning where the learner is allowed to choose the order of the datapoints in order to minimize their regret (aka self-directed online learning).  We focus on the fundamental task of online linear regression: the learner is given a dataset $X$ with $n$ examples in $d$ dimensions and at step $t$  they select a point $x_t \in X$, predict a value $\widetilde y_t$,  and suffer loss $(\widetilde y_t - w^\ast \cdot x_t)^2$.  The goal is to design algorithms that order the examples and achieve better 
regret  than random- or worst-order online algorithms.

For an arbitrary dataset $X$, we show that, under the Exponential Time Hypothesis, no efficient algorithm can approximate the optimal (best-order)  regret  within a factor of $d^{1/\poly(\log \log d)}$.


We then show that, for structured datasets, we can bypass the above hardness result and achieve nearly optimal regret. When the examples of $X$ are drawn i.i.d.\ from the uniform distribution on the sphere, we present an algorithm based on the greedy heuristic of  selecting ``easiest'' examples first that achieves a $\log d$-approximation of the optimal regret.
Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networks
Authors: Alexander Modell; Ian Gallagher; Emma Ceccherini; Nick Whiteley; Patrick Rubin-Delanchy
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We present a new representation learning framework, Intensity Profile Projection, for continuous-time dynamic network data. Given triples $(i,j,t)$, each representing a time-stamped ($t$) interaction between two entities ($i,j$), our procedure returns a continuous-time trajectory for each node, representing its behaviour over time. The framework consists of three stages: estimating pairwise intensity functions, e.g. via kernel smoothing; learning a projection which minimises a notion of intensity reconstruction error; and constructing evolving node representations via the learned projection. The trajectories satisfy two properties, known as structural and temporal coherence, which we see as fundamental for reliable inference. Moreoever, we develop estimation theory providing tight control on the error of any estimated trajectory, indicating that the representations could even be used in quite noise-sensitive follow-on analyses. The theory also elucidates the role of smoothing as a bias-variance trade-off, and shows how we can reduce the level of smoothing as the signal-to-noise ratio increases on account of the algorithm `borrowing strength' across the network.
The noise level in linear regression with dependent data
Authors: Ingvar Ziemann; Stephen Tu; George J. Pappas; Nikolai Matni
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We derive upper bounds for random design linear regression with dependent ($\beta$-mixing) data absent any realizability assumptions.  In contrast to the strictly realizable martingale noise regime, no sharp \emph{instance-optimal} non-asymptotics are available in the literature. Up to constant factors, our analysis correctly recovers the variance term predicted by the Central Limit Theorem---the noise level of the problem---and thus exhibits graceful degradation as we introduce misspecification. Past a burn-in, our result is sharp in the moderate deviations regime, and in particular does not inflate the leading order term by mixing time factors.
Optimistic Natural Policy Gradient: a Simple Efficient Policy Optimization Framework  for Online RL
Authors: Qinghua Liu; Gellért Weisz; András György; Chi Jin; Csaba Szepesvari
Venue: NeurIPS 2023 (NeurIPS 2023 spotlight)

Abstract: While policy optimization algorithms have played an important role in recent empirical success of Reinforcement Learning (RL), the existing theoretical understanding of policy optimization remains rather limited---they are either restricted to tabular MDPs or suffer from highly suboptimal sample complexity, especial in online RL where exploration is necessary. This paper proposes a simple efficient policy optimization framework---Optimistic NPG for online RL. Optimistic NPG can be viewed as simply combining of the classic natural policy gradient (NPG) algorithm [Kakade, 2001]  with optimistic policy evaluation subroutines to encourage exploration. For $d$-dimensional linear MDPs, Optimistic NPG is computationally efficient, and learns an $\epsilon$-optimal policy within  $\tilde{\mathcal{O}}(d^2/\epsilon^3)$ samples, which is the first computationally efficient algorithm whose sample complexity has the optimal dimension dependence $\tilde{\Theta}(d^2)$. It also improves over state-of-the-art results of policy optimization algorithms [Zanette et al., 2021] by a factor of $d$. For general function approximation that subsumes linear MDPs, Optimistic NPG, to our best knowledge, is also the first policy optimization algorithm that achieves the polynomial sample complexity for learning near-optimal policies.
MarioGPT: Open-Ended Text2Level Generation through Large Language Models
Authors: Shyam Sudhakaran; Miguel González-Duque; Matthias Freiberger; Claire Glanois; Elias Najarro; Sebastian Risi
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: Procedural Content Generation (PCG) is a technique to generate complex and diverse environments in an automated way. However, while generating content with PCG methods is often straightforward, generating meaningful content that reflects specific  intentions and constraints remains challenging. Furthermore, many PCG algorithms lack the ability to generate content in an open-ended manner. Recently, Large Language Models (LLMs) have  shown to be incredibly effective in many diverse domains. These trained LLMs can be fine-tuned, re-using information and accelerating training for new tasks. Here, we introduce MarioGPT, a fine-tuned GPT2 model trained to generate tile-based game levels, in our case Super Mario Bros levels. MarioGPT can not only generate diverse levels, but can be  text-prompted for controllable level generation, addressing one of the key challenges of current PCG techniques.  As far as we know, MarioGPT is the first text-to-level model and combined with novelty search it enables the  generation of diverse levels with varying play-style dynamics (i.e. player paths) and the open-ended discovery of an increasingly diverse range of content. 
Code available at https://github.com/shyamsn97/mario-gpt.
Distribution-Free Statistical Dispersion Control for Societal Applications
Authors: Zhun Deng; Thomas P Zollo; Jake Snell; Toniann Pitassi; Richard Zemel
Venue: NeurIPS 2023 (NeurIPS 2023 spotlight)

Abstract: Explicit finite-sample statistical guarantees on model performance are an important ingredient in responsible machine learning.  Previous work has focused mainly on bounding either the expected loss of a predictor or the probability that an individual prediction will incur a loss value in a specified range.  However, for many high-stakes applications it is crucial to understand and control the \textit{dispersion} of a loss distribution, or the extent to which different members of a population experience unequal effects of algorithmic decisions. We initiate the study of distribution-free control of statistical dispersion measures with societal implications and propose a simple yet flexible framework that allows us to handle a much richer class of statistical functionals beyond previous work. Our methods are verified through experiments in toxic comment detection, medical imaging, and film recommendation.
Small batch deep reinforcement learning
Authors: Johan Samir Obando Ceron; Marc G Bellemare; Pablo Samuel Castro
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: In value-based deep reinforcement learning with replay memories, the batch size parameter specifies how many transitions to sample for each gradient update. Although critical to the learning process, this value is typically not adjusted when proposing new algorithms. In this work we present a broad empirical study that suggests reducing the batch size can result in a number of significant performance gains; this is surprising, as the general tendency when training neural networks is towards larger batch sizes for improved performance. We complement our experimental findings with a set of empirical analyses towards better understanding this phenomenon.
High Precision Causal Model Evaluation with Conditional Randomization
Authors: Chao Ma; Cheng Zhang
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: The gold standard for causal model evaluation involves comparing model predictions with true effects estimated from randomized controlled trials (RCT). However, RCTs are not always feasible or ethical to perform. In contrast, conditionally randomized experiments based on inverse probability weighting (IPW) offer a more realistic approach but may suffer from high estimation variance. To tackle this challenge and enhance causal model evaluation in real-world conditional randomization settings, we introduce a novel low-variance estimator for causal error, dubbed as the pairs estimator. By applying the same IPW estimator to both the model and true experimental effects, our estimator effectively cancels out the variance due to IPW and achieves a smaller asymptotic variance. Empirical studies demonstrate the improved of our estimator, highlighting its potential on achieving near-RCT performance. Our method offers a simple yet powerful solution to evaluate causal inference models in conditional randomization settings without complicated modification of the IPW estimator itself, paving the way for more robust and reliable model assessments.
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference
Authors: Tao Lei; Junwen Bai; Siddhartha Brahma; Joshua Ainslie; Kenton Lee; Yanqi Zhou; Nan Du; Vincent Y Zhao; Yuexin Wu; Bo Li; Yu Zhang; Ming-Wei Chang
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We propose Conditional Adapter (CoDA), a parameter-efficient transfer learning method that also improves inference efficiency. CoDA generalizes beyond standard adapter approaches to enable a new way of balancing speed and accuracy using conditional computation.
Starting with an existing dense pretrained model, CoDA adds sparse activation together with a small number of new parameters and a light-weight training phase.
Our experiments demonstrate that the CoDA approach provides an unexpectedly efficient way to transfer knowledge.
Across a variety of language, vision, and speech tasks, CoDA achieves a 2x to 8x inference speed-up compared to the state-of-the-art Adapter approaches with moderate to no accuracy loss and the same parameter efficiency.
SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models
Authors: Martin Gonzalez; Nelson Fernandez; Thuy Vinh Dinh Tran; Elies Gherbi; Hatem Hajri; Nader Masmoudi
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: A potent class of generative models known as Diffusion Probabilistic Models
(DPMs) has become prominent. A forward diffusion process adds gradually noise
to data, while a model learns to gradually denoise. Sampling from pre-trained
DPMs is obtained by solving differential equations (DE) defined by the learnt
model, a process which has shown to be prohibitively slow. Numerous efforts on
speeding-up this process have consisted on crafting powerful ODE solvers.
Despite being quick, such solvers do not usually reach the optimal quality
achieved by available slow SDE solvers. Our goal is to propose SDE solvers that
reach optimal quality without requiring several hundreds or thousands of NFEs
to achieve that goal. We propose Stochastic Explicit Exponential
Derivative-free Solvers (SEEDS), improving and generalizing Exponential
Integrator approaches to the stochastic case on several frameworks. 
After carefully analyzing the formulation of exact
solutions of diffusion SDEs, we craft SEEDS to analytically compute the linear
part of such solutions. Inspired by the Exponential Time-Differencing method,
SEEDS use a novel treatment of the stochastic components of solutions,
enabling the analytical computation of their variance, and contains high-order
terms allowing to reach optimal quality sampling $\sim3$-$5\times$ faster than previous
SDE methods. We validate our approach on several image generation benchmarks,
showing that SEEDS outperform or are competitive with previous SDE solvers.
Contrary to the latter, SEEDS are derivative and training free, and we fully
prove strong convergence guarantees for them.
Gacs-Korner Common Information Variational Autoencoder
Authors: Michael Kleinman; Alessandro Achille; Stefano Soatto; Jonathan Kao
Venue: NeurIPS 2023 (NeurIPS 2023 poster)

Abstract: We propose a notion of common information that allows one to quantify and separate the information that is shared between two random variables from the information that is unique to each. Our notion of common information is defined by an optimization problem over a family of functions and recovers the G\'acs-K\"orner common information as a special case. Importantly, our notion can be approximated empirically using samples from the underlying data distribution. We then provide a method to partition and quantify the common and unique information using a simple modification of a traditional variational auto-encoder. Empirically, we demonstrate that our formulation allows us to learn semantically meaningful common and unique factors of variation even on high-dimensional data such as images and videos. Moreover, on datasets where ground-truth latent factors are known, we show that we can accurately quantify the common information between the random variables.