GenAI4ELab/papercli-papers huggingface dataset:
Uncovering the Effectiveness of Calibration on Open Intent Classification
Authors: Hyejin Won; Kyung Ho Park
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Open intent classification aims to simultaneously identify known and unknown intents, and it is one of the challenging tasks in modern dialogue systems. While prior approaches are based on known intent classifiers trained under the cross-entropy loss, we presume this loss function yields a representation overly biased to the known intents; thus, it negatively impacts identifying unknown intents. In this study, we propose a novel open intent classification approach that utilizes model calibration into the previously-proposed state-of-the-art. We empirically examine that simply changing a learning objective in a more calibrated manner outperforms the past state-of-the-art. We further excavate that the underlying reason behind calibrated classifier's supremacy derives from the high-level layers of the deep neural networks. We also discover that our approach is robust to harsh settings where few training samples per class exist. Consequentially, we expect our findings and takeaways to exhibit practical guidelines of open intent classification, thus helping to inform future model design choices.
Multi-Task Structural Learning using Local Task Similarity induced Neuron Creation and Removal
Authors: Naresh Kumar Gurulingan; Elahe Arani; Bahram Zonooz
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Multi-task learning has the potential to improve generalization by maximizing positive transfer between tasks while reducing task interference. Fully achieving this potential is hindered by manually designed architectures that remain static throughout training. In contrast, learning in the brain occurs through structural changes that are in tandem with changes in synaptic strength. Therefore, we propose Multi-Task Structural Learning (MTSL) which simultaneously learns the multi-task architecture and its parameters. MTSL begins with an identical single task network for each task and alternates between a task learning phase and a structural learning phase. In the task learning phase, each network specializes in the corresponding task. In each of the structural learning phases, starting from the earliest layer, locally similar task layers first transfer their knowledge to a newly created group layer after which they become redundant and are removed. Our experimental results show that MTSL achieves competitive generalization with various baselines and improves robustness to out-of-distribution data.
Prompt-to-Prompt Image Editing with Cross-Attention Control
Authors: Amir Hertz; Ron Mokady; Jay Tenenbaum; Kfir Aberman; Yael Pritch; Daniel Cohen-Or
Venue: ICLR 2023 (ICLR 2023 notable top 25%)

Abstract: Recent large-scale text-driven synthesis diffusion models have attracted much attention thanks to their remarkable capabilities of generating highly diverse images that follow given text prompts. Therefore, it is only natural to build upon these synthesis models to provide text-driven image editing capabilities. However, Editing is challenging for these generative models, since an innate property of an editing technique is to preserve some content from the original image, while in the text-based models, even a small modification of the text prompt often leads to a completely different outcome. State-of-the-art methods mitigate this by requiring the users to provide a spatial mask to localize the edit, hence, ignoring the original structure and content within the masked region. In this paper, we pursue an intuitive prompt-to-prompt editing framework, where the edits are controlled by text only. We analyze a text-conditioned model in depth and observe that the cross-attention layers are the key to controlling the relation between the spatial layout of the image to each word in the prompt. With this observation, we propose to control the attention maps along the diffusion process. Our approach enables us to monitor the synthesis process by editing the textual prompt only, paving the way to a myriad of caption-based editing applications such as localized editing by replacing a word, global editing by adding a specification, and even controlling the extent to which a word is reflected in the image. We present our results over diverse images and prompts with different text-to-image models, demonstrating high-quality synthesis and fidelity to the edited prompts.
Timing is Everything: Learning to Act Selectively with Costly Actions and Budgetary Constraints
Authors: David Henry Mguni; Aivar Sootla; Juliusz Krzysztof Ziomek; Oliver Slumbers; Zipeng Dai; Kun Shao; Jun Wang
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Many real-world settings involve costs for performing actions; transaction costs
in financial systems and fuel costs being common examples. In these settings,
performing actions at each time step quickly accumulates costs leading to vastly
suboptimal outcomes. Additionally, repeatedly acting produces wear and tear and
ultimately, damage. Determining when to act is crucial for achieving successful
outcomes and yet, the challenge of efficiently learning to behave optimally when
actions incur minimally bounded costs remains unresolved. In this paper, we intro-
duce a reinforcement learning (RL) framework named Learnable Impulse Control
Reinforcement Algorithm (LICRA), for learning to optimally select both when
to act and which actions to take when actions incur costs. At the core of LICRA
is a nested structure that combines RL and a form of policy known as impulse
control which learns to maximise objectives when actions incur costs. We prove
that LICRA, which seamlessly adopts any RL method, converges to policies that
optimally select when to perform actions and their optimal magnitudes. We then
augment LICRA to handle problems in which the agent can perform at most k < ∞
actions and more generally, faces a budget constraint. We show LICRA learns the
optimal value function and ensures budget constraints are satisfied almost surely.
We demonstrate empirically LICRA’s superior performance against benchmark
RL methods in OpenAI gym’s Lunar Lander and in Highway environments and a
variant of the Merton portfolio problem within finance.
Closing the Gap Between SVRG and TD-SVRG with Gradient Splitting
Authors: Arsenii Mustafin; Ioannis Paschalidis; Alex Olshevsky
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Temporal difference (TD) learning is a simple algorithm for policy evaluation
in reinforcement learning. The performance of TD learning is affected by high
variance and it can be naturally enhanced with variance reduction techniques, such
as the Stochastic Variance Reduced Gradient (SVRG) method. Recently, multiple
works have sought to fuse TD learning with SVRG to obtain a policy evaluation
method with a linear rate of convergence. However, the resulting convergence rate
is significantly weaker than what is achieved by SVRG in the setting of convex
optimization. In this work we utilize a recent interpretation of TD-learning as the
splitting of the gradient of an appropriately chosen function, thus simplifying the
algorithm and fusing TD with SVRG. We prove a linear convergence bound that
is identical to the convergence bound available for SVRG in the convex setting.
Globally Optimal Training of Neural Networks with Threshold Activation Functions
Authors: Tolga Ergen; Halil Ibrahim Gulluk; Jonathan Lacotte; Mert Pilanci
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Threshold activation functions are highly preferable in neural networks due to their efficiency in hardware implementations. Moreover, their mode of operation is more interpretable and resembles that of biological neurons. However, traditional gradient based algorithms such as Gradient Descent cannot be used to train the parameters of neural networks with threshold activations since the activation function has zero gradient except at a single non-differentiable point. To this end, we study weight decay regularized training problems of deep neural networks with threshold activations. We first show that regularized deep threshold network training problems can be equivalently formulated as a standard convex optimization problem, which parallels the LASSO method, provided that the last hidden layer width exceeds a certain threshold. We also derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network. We corroborate our theoretical results with various numerical experiments.
SYNC: SAFETY-AWARE NEURAL CONTROL FOR STABILIZING STOCHASTIC DELAY-DIFFERENTIAL EQUATIONS
Authors: Jingdong Zhang; Qunxi Zhu; Wei Yang; Wei Lin
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Stabilization of the systems described by \textit{stochastic delay}-differential equations (SDDEs) under preset conditions is a challenging task in the control community. Here, to achieve this task, we leverage neural networks to learn control policies using the information of the controlled systems in some prescribed regions.  Specifically, two learned control policies, i.e., the neural deterministic controller (NDC) and the neural stochastic controller (NSC), work effectively in the learning procedures that rely on, respectively, the well-known LaSalle-type theorem and the newly-established theorem for guaranteeing the stochastic stability in SDDEs. We theoretically investigate the performance of the proposed controllers in terms of convergence time and energy cost.  More practically and significantly, we improve our learned control policies through considering the situation where the controlled trajectories only evolve in some specific safety set. {\color{black}  The practical validity of such control policies restricted in safety set is attributed to the theory that we further develop for safety and stability guarantees in SDDEs using the stochastic control barrier function and the spatial discretization}. We call this control as SYNC (\textbf{S}afet\textbf{Y}-aware \textbf{N}eural \textbf{C}ontrol).   The efficacy of all the articulated control policies, including the SYNC, is demonstrated systematically by using representative control problems.
CAMVR: Context-Adaptive Multi-View Representation Learning for Dense Retrieval
Authors: zhilin liao; XIANGTING HOU; Dongfang Lou; Ningyu Zhang; Huajun Chen
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: The recently proposed MVR (Multi-View Representation) model achieves remarkable performance in open-domain dense retrieval. In MVR, the document can match with multi-view queries by encoding the document into multiple representations. However, these representations tend to collapse into the same one when the percentage of documents answering multiple queries in training data is low. In this paper, we propose a CAMVR (Context-Adaptive Multi-View Representation) learning framework, which explicitly avoids the collapse problem by aligning each viewer token with different document snippets. In CAMVR, each viewer token is placed before each snippet to capture the local and global information with the consideration that answers of different view queries may scatter in one document. In addition, the view of the snippet containing the answer is used to explicitly supervise the learning process, from which the interpretability of view representation is provided. The extensive experiments show that CAMVR outperforms the existing models and achieves state-of-the-art results.
FedAvg Converges to Zero Training Loss Linearly: The Power of Overparameterized Multi-Layer Neural Networks
Authors: Bingqing Song; Prashant Khanduri; Xinwei Zhang; Jinfeng Yi; Mingyi Hong
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Federated Learning (FL) is a distributed learning paradigm that allows multiple clients to learn a joint model by utilizing privately held data at each client. Significant research efforts have been devoted to develop advanced algorithms that deal with the situation where the data at individual clients have different distributions (i.e., the data heterogeneity issue). In this work, we show that data heterogeneity can be dealt from a different perspective. That is, by utilizing a certain overparameterized multi-layer neural network at each client, even the vanilla FedAvg (a.k.a. the Local SGD) algorithm can accurately optimize the training problem. Specifically, when each client has a neural network with one wide layer of size $N$ (where $N$ is the number of total training samples), followed by layers of smaller widths, FedAvg converges linearly to a solution that achieves (almost) zero training loss, without requiring any assumptions on the data distributions at each client. To our knowledge, this is the first work that demonstrates such resilience to data heterogeneity
for FedAvg when trained on multi-layer neural networks. Our experiments also confirm that, neural network of large size can achieve better and more stable performance for FL problems.
A Simulation-based Framework for Robust Federated Learning to Training-time Attacks
Authors: Wanyun Xie; Thomas Pethick; Ali Ramezani-Kebrya; Volkan Cevher
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Well-known robust aggregation schemes in federated learning (FL) are shown to be vulnerable to an informed adversary who can tailor training-time attacks [Fang et al., Xie et al.]. We frame robust distributed learning problem as a game between a server and an adversary that is able to optimize strong training-time attacks. We introduce RobustTailor, a simulation-based framework that prevents the adversary from being omniscient. The simulated game we propose enjoys theoretical guarantees through a regret analysis. RobustTailor improves robustness to training-time attacks significantly while preserving almost the same privacy guarantees as standard robust aggregation schemes in FL. Empirical results under challenging attacks show that RobustTailor performs similar to an upper bound with perfect knowledge of honest clients.
Continuous-time identification of dynamic state-space models by deep subspace encoding
Authors: Gerben I. Beintema; Maarten Schoukens; Roland Tóth
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Continuous-time (CT) modeling has proven to provide improved sample efficiency and interpretability in learning the dynamical behavior of physical systems compared to discrete-time (DT) models. However, even with numerous recent developments, the CT nonlinear state-space (NL-SS) model identification problem remains to be solved in full, considering common experimental aspects such as the presence of external inputs, measurement noise, latent states, and general robustness. This paper presents a novel estimation method that addresses all these aspects and that can obtain state-of-the-art results on multiple benchmarks with compact fully connected neural networks capturing the CT dynamics. The proposed estimation method called the subspace encoder approach (SUBNET) ascertains these results by efficiently approximating the complete simulation loss by evaluating short simulations on subsections of the data, by using an encoder function to estimate the initial state for each subsection and a novel state-derivative normalization to ensure stability and good numerical conditioning of the training process. We prove that the use of subsections increases cost function smoothness together with the necessary requirements for the existence of the encoder function and we show that the proposed state-derivative normalization is essential for reliable estimation of CT NL-SS models.
RainProof: An Umbrella to Shield Text Generator from Out-Of-Distribution Data
Authors: Maxime DARRIN; Pablo Piantanida; Pierre Colombo
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: As more and more conversational and translation systems are deployed in production, it is essential to implement and develop effective control mechanisms to ensure their proper functioning and security. An essential component to ensure the safe behavior of the system is out-of-distribution (OOD) detection, which aims to detect whether an input sample is statistically far from the training distribution. While OOD detection is a widely covered topic in classification tasks, it has received much less attention in text generation. This paper addresses the problem of OOD detection for machine translation and dialog generation from an operational perspective. Our contribution includes (i) RAINPROOF a Relative informAItioN Projection Out OF distribution detection framework and (ii) a more operational evaluation setting for OOD detection. Surprisingly, we find that OOD detection is not necessarily aligned with task-specific measures. The OOD detector may filter out samples that are well processed by the model and keep samples that are not, leading to weaker performance. Our results show that RAINPROOF breaks this curse and achieve good results in OOD detection while increasing system performance.
Off Policy Average Reward Actor Critic with Deterministic Policy Search
Authors: Naman Saxena; Subhojyoti Khastagir; Shishir N Y; Shalabh Bhatnagar
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: The average reward criterion is relatively less explored as most existing works in the Reinforcement Learning literature consider the discounted reward criterion. There are few recent works that present on-policy average reward actor-critic algorithms, but average reward off-policy actor-critic is relatively less explored. In this paper, we present both on-policy and off-policy deterministic policy gradient theorems for the average reward performance criterion. Using these theorems, we also present an Average Reward Off-Policy Deep Deterministic Policy Gradient (ARO-DDPG) Algorithm. We show a finite time analysis of the resulting three-timescale stochastic approximation scheme and obtain an $\epsilon$-optimal stationary policy with a sample complexity of $\Omega(\epsilon^{-2.5})$. We compare the average reward performance of our proposed algorithm and observe better empirical performance compared to state-of-the-art on-policy average reward actor-critic algorithms over MuJoCo based environments.
Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction
Authors: David Klee; Ondrej Biza; Robert Platt; Robin Walters
Venue: ICLR 2023 (ICLR 2023 notable top 5%)

Abstract: Predicting the pose of objects from a single image is an important but difficult computer vision problem. Methods that predict a single point estimate do not predict the pose of objects with symmetries well and cannot represent uncertainty. Alternatively, some works predict a distribution over orientations in $\mathrm{SO}(3)$. However, training such models can be computation- and sample-inefficient. Instead, we propose a novel mapping of features from the image domain to the 3D rotation manifold. Our method then leverages $\mathrm{SO}(3)$ equivariant layers, which are more sample efficient, and outputs a distribution over rotations that can be sampled at arbitrary resolution. We demonstrate the effectiveness of our method at object orientation prediction, and achieve state-of-the-art performance on the popular PASCAL3D+ dataset. Moreover, we show that our method can model complex object symmetries, without any modifications to the parameters or loss function.  Code is available at \url{https://dmklee.github.io/image2sphere}.
Weakly-Supervised Domain Adaptation in Federated Learning
Authors: Enyi Jiang; Oluwasanmi O Koyejo
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Federated domain adaptation (FDA) describes the setting where a set of source clients seek to optimize the performance of a target client. To be effective, FDA must address some of the distributional challenges of Federated learning (FL). For instance, FL systems exhibit distribution shifts across clients. Further, labeled data are not always available among the clients. To this end, we propose and compare novel approaches for FDA, combining the few labeled target samples with the source data when auxiliary labels are available to the clients. The in-distribution auxiliary information is included during local training to boost out-of-domain accuracy. Also, during fine-tuning, we devise a simple yet efficient gradient projection method to detect the valuable components from each source client model towards the target direction. The extensive experiments on medical imaging datasets show that our proposed framework significantly improves federated domain adaptation performance.
Computational Doob h-transforms for Online Filtering of Discretely Observed Diffusions
Authors: Nicolas Chopin; Andras Fulop; Jeremy Heng; Alexandre H. Thiery
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: This paper is concerned with online filtering of discretely observed nonlinear diffusion processes. Our approach is based on the fully adapted auxiliary particle filter, which involves Doob's $h$-transforms that are typically intractable. We propose a computational framework to approximate these $h$-transforms by solving the underlying backward Kolmogorov equations using nonlinear Feynman-Kac formulas and neural networks. The methodology allows one to train a locally optimal particle filter prior to the data-assimilation procedure. Numerical experiments illustrate that the proposed approach can be orders of magnitude more efficient than the bootstrap particle filter in the regime of highly informative observations, when the observations are extreme under the model, and if the state dimension is large.
Concept Gradient: Concept-based Interpretation Without Linear Assumption
Authors: Andrew Bai; Chih-Kuan Yeh; Neil Y.C. Lin; Pradeep Kumar Ravikumar; Cho-Jui Hsieh
Venue: ICLR 2023 (ICLR 2023 poster)

Abstract: Concept-based interpretations of black-box models are often more intuitive for humans to understand. The most widely adopted approach for concept-based, gradient interpretation is Concept Activation Vector (CAV). CAV relies on learning a linear relation between some latent representation of a given model and concepts. The premise of meaningful concepts lying in a linear subspace of model layers is usually implicitly assumed but does not hold true in general. In this work we proposed Concept Gradient (CG), which extends concept-based, gradient interpretation methods to non-linear concept functions. We showed that for a general (potentially non-linear) concept, we can mathematically measure how a small change of concept affects the model’s prediction, which is an extension of gradient-based interpretation to the concept space. We demonstrated empirically that CG outperforms CAV in attributing concept importance on real world datasets and performed case study on a medical dataset. The code is available at github.com/jybai/concept-gradients.
Would decentralization hurt generalization?
Authors: Tongtian Zhu; Fengxiang He; Kaixuan Chen; Mingli Song; Dacheng Tao
Venue: ICLR 2023 (Submitted to ICLR 2023)

Abstract: Decentralized stochastic gradient descent (D-SGD) allows collaborative learning on massive devices without controlling of a central server. Existing theory suggests that the decentralization degrades the generalizability, which conflicts with experimental results in the large-batch settings that D-SGD generalize better than centralized SGD (C-SGD). This work presents new theory that reconciles the conflict between the two perspectives. We prove that D-SGD introduces an implicit regularization that simultaneously penalizes (1) the sharpness of the learned minima and (2) the consensus distance between the consensus model and local models. We then prove that the implicit regularization is amplified in the large-batch settings when the linear scaling rule is applied. We further analyze the escaping efficiency of D-SGD, which suggests that D-SGD favors super-quadratic flat minima. Experiments are in full agreement with our theory. The code will be released publicly. To our best knowledge, this is the first work on the implicit regularization and escaping efficiency of D-SGD.