pwc-archive/papers-with-abstracts huggingface dataset:
Bootstrapped Hindsight Experience replay with Counterintuitive Prioritization
Authors: Jiawei Xu, Shuxing Li, Chun Yuan, Zhengyou Zhang, Lei Han
Date: 2021-09-29 00:00:00
Tasks: Q-Learning

Goal-conditioned environments are known as sparse rewards tasks, in which the agent gains a positive reward only when it achieves the goal. Such an setting results in much difficulty for the agent to explore successful trajectories. Hindsight experience replay (HER) replaces the goal in failed experiences with any practically achieved one, so that the agent has a much higher chance to see successful trajectories even if they are fake. Comprehensive results have demonstrated the effectiveness of HER in the literature. However, the importance of the fake trajectories differs in terms of exploration and exploitation, and it is usually inefficient to learn with a fixed proportion of fake and original data as HER did. In this paper, inspired by Bootstrapped DQN, we use multiple heads in DDPG and take advantage of the diversity and uncertainty among multiple heads to improve the data efficiency with relabeled goals. The method is referred to as Bootstrapped HER (BHER). Specifically, in addition to the benefit from the Bootstrapped version, we explicitly leverage the uncertainty measured by the variance of estimated Q-values from multiple heads. A common knowledge is that higher uncertainty will promote exploration and hence maximizing the uncertainty via a bonus term will induce better performance in Q-learning. However, in this paper, we reveal a counterintuitive conclusion that for hindsight experiences, exploiting lower uncertainty data samples will significantly improve the performance. The explanation behind this fact is that hindsight relabeling largely promotes exploration, and then exploiting lower uncertainty data (whose goals are generated by hindsight relabeling) provides a good trade-off between exploration and exploitation, resulting in further improved data efficiency. Comprehensive experiments demonstrate that our method can achieve state-of-the-art results in many goal-conditioned tasks.
Physical System Design Using Hamiltonian Monte Carlo over Learned Manifolds
Authors: Adam D. Cobb, Anirban Roy, Kaushik Koneripalli, Daniel Elenius, Susmit Jha
Date: 2021-09-29 00:00:00
Tasks: valid

The design of complex physical systems entails satisfying several competing performance objectives. In practice, some design requirements are often implicit in the intuition and knowledge of designers who have many years of experience working with similar designs. Designers use this experience to sample a few promising candidates in the design space and evaluate or simulate them using detailed, typically slow multiphysics models. The goal in design is usually to generate a diverse set of high-performing design configurations that allow trade-offs across different objectives and avoid early concretization. In this paper, we develop a machine learning approach to automate physical system design. We use deep generative models to learn a manifold of the valid design space, followed by Hamiltonian Monte Carlo (HMC) with simulated annealing to explore and optimize design over the learned manifold, producing a diverse set of optimal designs. Our approach is akin to partial simulated annealing restricted to the learned design manifold, where the annealing schedule is varied to trade-off different objectives. To prevent our approach from traversing off the design manifold and proposing unreliable designs, we leverage Monte Carlo dropout as a way to detect and avoid design configurations where the learned model cannot be trusted. We demonstrate the efficacy of our proposed approach using several case studies that include the design of an SAE race vehicle, propeller, and air vehicle. Across these case studies, we successfully show how our method generates high-performing and diverse designs.
Calibration Regularized Training of Deep Neural Networks using Kernel Density Estimation
Authors: Teodora Popordanoska, Raphael Sayer, Matthew B. Blaschko
Date: 2021-09-29 00:00:00
Tasks: Autonomous Driving, Density Estimation, Medical Diagnosis

Calibrated probabilistic classifiers are models whose predicted probabilities can directly be interpreted as uncertainty estimates. This property is particularly important in safety-critical applications such as medical diagnosis or autonomous driving. However, it has been shown recently that deep neural networks are poorly calibrated and tend to output overconfident predictions. As a remedy, we propose a trainable calibration error estimator based on Dirichlet kernel density estimates, which asymptotically converges to the true Lp calibration error. This novel estimator enables us to achieve the strongest notion of multiclass calibration, called canonical calibration, while other common calibration methods only allow for top-label and marginal calibration. The empirical results show that our estimator is competitive with the state-of-the-art, consistently yielding tradeoffs between calibration error and accuracy that are (near) Pareto optimal across a range of network architectures.  The computational complexity of our estimator is O(n^2), matching that of the kernel maximum mean discrepancy, used in a previously considered trainable calibration estimator. By contrast, the proposed method has a natural choice of kernel, and can be used to generate consistent estimates of other quantities based on conditional expectation, such as the sharpness of an estimator.
Fine-Tuning from Limited Feedbacks
Authors: Jing Li, Yuangang Pan, Yueming Lyu, Yinghua Yao, Ivor Tsang
Date: 2021-09-29 00:00:00
Tasks: Fairness

Instead of learning from scratch, fine-tuning a pre-trained model to fit a related target dataset of interest or downstream tasks has been a handy trick to achieve the desired performance. However, according to the study of~\cite{song2017machine}, standard fine-tuning may expose the information about target data if the pre-trained model is supplied by a malicious provider. Instead of reckoning that data holders are always expert to select reliable models and execute fine-tuning themselves, this paper confronts this problem by exploring a new learning paradigm named Fine-Tuning from limited FeedBacks (FTFB). The appealing trait of FTFB is that the model tuning does not require directly seeing the target data but leveraging the model performances as feedbacks instead. To learn from query-feedback, we propose to fine-tune the pre-trained model on the parameter distribution with the gradient descent scheme. For the deep models whose tuning parameters distribute across multiple layers, a more query-efficient algorithm is further designed which refines the model layer by layer sequentially with importance weight. Extensive experiments on various tasks demonstrate that the proposed algorithms significantly improve the pre-trained model with limited feedbacks only. For downstream tasks which adopt inconsistent evaluation measurement with pre-training, such as fairness or fault-intolerance, we verify our algorithms can also reach good performance.
ConFeSS: A Framework for Single Source Cross-Domain Few-Shot Learning
Authors: Debasmit Das, Sungrack Yun, Fatih Porikli
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Contrastive Learning, Cross-Domain Few-Shot, cross-domain few-shot learning, domain classification, feature selection, Few-Shot Learning, Meta-Learning

Most current few-shot learning methods train a model from abundantly labeled base category data and then transfer and adapt the model to sparsely labeled novel category data. These methods mostly generalize well on novel categories from the same domain as the base categories but perform poorly for distant domain categories. In this paper, we propose a framework for few-shot learning coined as ConFeSS (Contrastive Learning and Feature Selection System) that tackles large domain shift between base and novel categories. The first step of our framework trains a feature extracting backbone with the contrastive loss on the base category data. Since the contrastive loss does not use supervision, the features can generalize better to distant target domains. For the second step, we train a masking module to select relevant features that are more suited to target domain classification. Finally, a classifier is fine-tuned along with the backbone such that the backbone produces features similar to the relevant ones. To evaluate our framework, we tested it on a recently introduced cross-domain few-shot learning benchmark. Experimental results demonstrate that our framework outperforms all meta-learning approaches and produces competitive results against recent cross-domain methods. Additional analyses are also performed to better understand our framework.
Neural Combinatorial Optimization with Reinforcement Learning : Solving theVehicle Routing Problem with Time Windows
Authors: Abdelhakim Abdellaoui, Issmail El Hallaoui, Loubna Benabbou
Date: 2021-09-29 00:00:00
Tasks: Combinatorial Optimization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In contrast to the classical techniques for solving combinatorial optimization problems, recent advancements in reinforcement learning yield the potential to independently learn heuristics without any human interventions. In this context, the current paper aims to present a complete framework for solving the vehicle routing problem with time windows (VRPTW) relying on neural networks and reinforcement learning. Our approach is mainly based on an attention model (AM) that predicts the near-optimal distribution over different problem instances. To optimize its parameters, this model is trained in a reinforcement learning(RL) environment using a stochastic policy gradient and through a real-time evaluation of the reward, quantity to meet the problem business and logical constraints. Using synthetic data, the proposed model outperforms some existing baselines. This performance comparison was on the basis of the solution quality (total tour length) and the computation time (inference time) for small and medium-sized samples.
Preference Conditioned Neural Multi-objective Combinatorial Optimization
Authors: Xi Lin, Zhiyuan Yang, Qingfu Zhang
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: Combinatorial Optimization, Traveling Salesman Problem

Multiobjective combinatorial optimization (MOCO) problems can be found in many real-world applications. However, exactly solving these problems would be very challenging, particularly when they are NP-hard. Many handcrafted heuristic methods have been proposed to tackle different MOCO problems over the past decades. In this work, we generalize the idea of neural combinatorial optimization, and develop a learning-based approach to approximate the whole Pareto set for a given MOCO problem without further search procedure. To be concrete, we propose a single preference-based attention model to directly generate approximate Pareto solutions of all the different trade-offs. We design an efficient multiobjective reinforcement learning algorithm to train the model with different preferences simultaneously. Experimental results show that our proposed method significantly outperforms the other methods on the multiobjective traveling salesman problem (MOTSP), multiobjective vehicle routing problem (MOVRP) and multiobjective knapsack problem (MOKP) in solution quality, speed, and model efficiency.
Interpreting Graph Neural Networks via Unrevealed Causal Learning
Authors: WanYu Lin, Hao Lan, Hao Wang, Baochun Li
Date: 2021-09-29 00:00:00
Tasks: Graph Learning

This paper proposes a new explanation framework, called OrphicX, for generating causal explanations for any graph neural networks (GNNs) based on learned latent causal factors. Specifically, we construct a distinct generative model and design an objective function that encourages the generative model to produce causal, compact, and faithful explanations. This is achieved by isolating the causal factors in the latent space of graphs by maximizing the information flow measurements. We theoretically analyze the cause-effect relationships in the proposed causal graph, identify node attributes as confounders between graphs and GNN predictions, and circumvent such confounder effect by leveraging the backdoor adjustment formula. Our framework is compatible with any GNNs, and it does not require access to the process by which the target GNN produces its predictions. In addition, it does not rely on the linear-independence assumption of the explained features, nor require prior knowledge on the graph learning tasks. Empirically, we show that OrphicX can effectively identify the causal semantics for generating causal explanations, significantly outperforming its alternatives.
Quantized sparse PCA for neural network weight compression
Authors: Andrey Kuzmin, Mart van Baalen, Markus Nagel, Arash Behboodi
Date: 2021-09-29 00:00:00
Tasks: Quantization

In this paper, we introduce a novel method of weight compression. In our method, we store weight tensors as sparse, quantized matrix factors, whose product is computed on the fly during inference to generate the target model's weight tensors. The underlying matrix factorization problem can be considered as a quantized sparse PCA problem and solved through iterative projected gradient descent methods. Seen as a unification of weight SVD, vector quantization and sparse PCA, our method achieves or is on par with state-of-the-art trade-offs between accuracy and model size. Our method is applicable to both moderate compression regime, unlike vector quantization, and extreme compression regime.
Label Refining: a semi-supervised method to extract voice characteristics without ground truth
Authors: Mathias Quillot, Richard Dufour, Jean-français Bonastre
Date: 2021-09-29 00:00:00

A characteristic is a distinctive trait shared by a group of observations which may be used to identify them. In the context of voice casting for audiovisual productions, characteristic extraction has an important role since it can help explaining the decisions of a voice recommendation system, or give modalities to the user with the aim to express a voice search request. Unfortunately, the lack of standard taxonomy to describe comedian voices prevents the implementation of an annotation protocol. To address this problem, we propose a new semi-supervised learning method entitled Label Refining that consists in extracting refined labels (e.g. vocal characteristics) from known initial labels (e.g. character played in a recording). Our proposed method first suggests using a representation extractor based on the initial labels, then computing refined labels using a clustering algorithm to finally train a refined representation extractor. The method is validated by applying Label Refining on recordings from the video game MassEffect 3. Experiments show that, using a subsidiary corpus, it is possible to bring out interesting voice characteristics without any a priori knowledge.
Revisiting Out-of-Distribution Detection: A Simple Baseline is Surprisingly Effective
Authors: Julian Bitterwolf, Alexander Meinke, Maximilian Augustin, Matthias Hein
Date: 2021-09-29 00:00:00
Tasks: Out-of-Distribution Detection, Out of Distribution (OOD) Detection

It is an important problem in trustworthy machine learning to recognize out-of-distribution (OOD) inputs which are inputs unrelated to the in-distribution task. Many out-of-distribution detection methods have been suggested in recent years. The goal of this paper is to recognize common objectives as well as to identify the implicit scoring functions of different OOD detection methods. In particular, we show that binary discrimination between in- and (different) out-distributions is equivalent to several different formulations of the OOD detection problem. When trained in a shared fashion with a standard classifier, this binary discriminator reaches an OOD detection performance similar to that of Outlier Exposure. Moreover, we show that the confidence loss which is used by Outlier Exposure has an implicit scoring function which differs in a non-trivial fashion from the theoretically optimal scoring function in the case where training and test out-distribution are the same, but is similar to the one used when training with an extra background class. In practice, when trained in exactly the same way, all these methods perform similarly and reach state-of-the-art OOD detection performance.
A Johnson-Lindenstrauss Framework for Randomly Initialized CNNs
Authors: Ido Nachum, Jan Hazla, Michael Gastpar, Anatoly Khina
Date: 2021-09-29 00:00:00
Proceeding: ICLR 2022 4
Tasks: LEMMA

How does the geometric representation of a dataset change after the application of each randomly initialized layer of a neural network? The celebrated Johnson-Lindenstrauss lemma answers this question for linear fully-connected neural networks (FNNs), stating that the geometry is essentially preserved. For FNNs with the ReLU activation, the angle between two input contracts according to a known mapping. The question for non-linear convolutional neural networks (CNNs) becomes much more intricate. To answer this question, we introduce a geometric framework. For linear CNNs, we show that the Johnson--Lindenstrauss lemma continues to hold, namely, that the angle between two inputs is preserved. For CNNs with ReLU activation, on the other hand, the behavior is richer: The angle between the outputs contracts, where the level of contraction depends on the nature of the inputs. In particular, after one layer, the geometry of natural images is essentially preserved, whereas for Gaussian correlated inputs, CNNs exhibit the same contracting behavior as FNNs with ReLU activation.
Optimized Separable Convolution: Yet Another Efficient Convolution Operator
Authors: Tao Wei, Yonghong Tian, YaoWei Wang, Yun Liang, Chang Wen Chen
Date: 2021-09-29 00:00:00

The convolution operation is the most critical component in recent surge of deep learning research. Conventional 2D convolution needs O(C^{2}K^{2}) parameters to represent, where C is the channel size and K is the kernel size. The amount of parameters has become really costly considering that these parameters increased tremendously recently to meet the needs of demanding applications. Among various implementations of the convolution, separable convolution has been proven to be more efficient in reducing the model size. For example, depth separable convolution reduces the complexity to O(C\cdot(C+K^{2})) while spatial separable convolution reduces the complexity to O(C^{2}K). However, these are considered ad hoc designs which cannot ensure that they can in general achieve optimal separation. In this research, we propose a novel and principled operator called optimized separable convolution by optimal design for the internal number of groups and kernel sizes for general separable convolutions can achieve the complexity of O(C^{\frac{3}{2}}K). When the restriction in the number of separated convolutions can be lifted, an even lower complexity at O(C\cdot\log(CK^{2})) can be achieved. Experimental results demonstrate that the proposed optimized separable convolution is able to achieve an improved performance in terms of accuracy-#Params trade-offs over both conventional, depth-wise, and depth/spatial separable convolutions.
ConVAEr: Convolutional Variational AutoEncodeRs for incremental similarity learning
Authors: Jiahao Huo, Terence L van Zyl
Date: 2021-09-29 00:00:00
Tasks: Metric Learning, Transfer Learning

Due to catastrophic forgetting, incremental similarity learning in neural networks remains an open challenge.   Previous  work  has  shown  that  keeping image  exemplars during  incremental  similarity  learning  is  effective  for  preserving  base knowledge (past learnt features and embeddings).   It is also generally accepted that the output layers learn more task-specific feature embeddings during the later training stages compared to the input layers’ general features earlier on. Building on these insights, we start by freezing the input layers of a neural network.  We then investigate the viability of generating “embedding” exemplars from a VAE that can protect base knowledge in the intermediate to output layers of the neural networks.  These generated exemplars replace the necessity for retaining images from previously learned classes. We experimented with three metric learning loss functions on the CUB-200 and CARS-196 in an incremental similarity learning setup.  We train different VAEs to generate exemplars from the intermediate convolution layers and linear output layers. We use these generated exemplars to rep-resent base knowledge. We compared our work to a previous technique that stores image exemplars.  The comparison is done for base knowledge, new knowledge and average knowledge preservation as metrics. The results show that generating exemplars from the linear and convolutional layers retained the highest ratio of base knowledge.  We note that using embeddings from the linear layers leads to better performance on new knowledge than convolutional embeddings.  Overall our methods yield better average knowledge performance across all experiments. These results support the view that for incremental similarity learning to overcome catastrophic forgetting, emphasis can be placed on learning embedding exemplars for intermediate to output layers. Further, we note that most incremental similarity learning for new classes depends on the linear layers rather than the convolutions. Further investigation into the relationship between transfer learning and similarity learning and the protection of intermediate layer embedding space for catastrophic forgetting is required.
Decentralized Cooperative Multi-Agent Reinforcement Learning with Exploration
Authors: Weichao Mao, Tamer Basar, Lin Yang, Kaiqing Zhang
Date: 2021-09-29 00:00:00
Tasks: Multi-agent Reinforcement Learning, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robot Navigation

Many real-world applications of multi-agent reinforcement learning (RL), such as multi-robot navigation and decentralized control of cyber-physical systems, involve the cooperation of agents as a team with aligned objectives. We study multi-agent RL in the most basic cooperative setting --- Markov teams --- a class of Markov games where the cooperating agents share a common reward. We propose an algorithm in which each agent independently runs stage-based V-learning (a Q-learning style algorithm) to efficiently explore the unknown environment, while using a stochastic gradient descent (SGD) subroutine for policy updates. We show that the agents can learn an $\epsilon$-approximate Nash equilibrium policy in at most $\propto\widetilde{O}(1/\epsilon^4)$ episodes. Our results advocate the use of a novel \emph{stage-based} V-learning approach to create a stage-wise stationary environment. We also show that under certain smoothness assumptions of the team, our algorithm can achieve a nearly \emph{team-optimal} Nash equilibrium. Simulation results corroborate our theoretical findings. One key feature of our algorithm is being \emph{decentralized}, in the sense that each agent has access to only the state and its local actions, and is even \emph{oblivious} to the presence of the other agents. Neither communication among teammates nor coordination by a central controller is required during learning. Hence, our algorithm can readily generalize to an arbitrary number of agents, without suffering from the exponential dependence on the number of agents.